0

I am writing a small web scraper in C#. It is going to use HttpWebRequest to get the HTML file, find the required data, then report it back to the caller.

The required data is only available when a user is logged in. As I am new to interfacing programmatically with http, Javascript, et al, I am not going to try and log on programmatically. The user will have to log on to the website, and my program will get the stored cookie, and load it into the CookieContainer for the http request.

I've done enough research to know that the data belongs in the CookieContainer (I think), but I can't seem to find anywhere an example of how to find a cookie created by IE (or firefox, or chrome, etc), load it programmatically, populate the CookieContainer, and send it with an http get request. So how does one do all that?

Thanks!

Steve H.
  • 3,283
  • 4
  • 24
  • 32

1 Answers1

2

I'm afraid you can't do that. Main reason is security. Because of cookies being used to identify a user, browser can't provide an easy access to cookies. Otherwise it would be really easy to still them.

You should better learn how to login user with HttpWebRequest or any other class like that.

Viktor S.
  • 12,736
  • 1
  • 27
  • 52
  • 1) Not true, using Fiddler I've read cookie values and hard coded them in last night. I am using a pre-existing session programatically. Now it's just a matter of finding the cookie file programmatically and populating the variables. 2) I would still like to know the proper way (login as user with HttpWebRequest) - please add link to answer and you'll get accepted. Thanks! – Steve H. Apr 02 '13 at 13:14
  • Fiddler is a proxy, that is why you can see it. It allows you to do a "man in the middle" attack. It has no access to cookies stored in browser - only an access to all HTTP requests happening in system and it can see only cookies sent with some request only when it is enabled. – Viktor S. Apr 02 '13 at 13:18
  • And for code: take a look at [this](http://stackoverflow.com/a/1274137/1558311) answer (one with 15 upvotes). You need code like that (except that parameter names will be different) and just take a cookie you need from `HttpWebResponse objResponse` – Viktor S. Apr 02 '13 at 13:23