c# httpwebrequest credential problem

Question

I am trying to login into www.diary.com using a httpwebrequest object. However, it always fail to login, and kept giving me back the login page. Can anyone enlighten me on what is/are wrong?

My code is as follows:

// prepare the web page we will be asking for
HttpWebRequest request = (HttpWebRequest)
    WebRequest.Create(@"http://diary.com/events/agenda");

request.ContentType = "text/html";

request.Credentials = new NetworkCredential(@"user@hotmail.com", "password");

request.AllowAutoRedirect = true;
request.Referer = @"http://diary.com/";

// execute the request
HttpWebResponse response = (HttpWebResponse)
    request.GetResponse();

// we will read data via the response stream
Stream resStream = response.GetResponseStream();

// set the WebBrowser object documentStream to the response stream
myWB.DocumentStream = resStream;

// simply tell me the title of the webpage
MessageBox.Show(myWB.Document.Title);

Answer 1

You have two problems here:

1. You are providing credentials at the protocol level, which is not how most websites (including this one) work. The protocol is totally anonymous, and the site uses Forms Authentication to log you in. Your code needs to actually create a POST request that mimics submitting the login form. The response that comes back from the server will include a cookie that has your auth token, which leads into...

2. You need to persist cookies across requests. After you submit the login request and get the cookie, you'll need to hang on to it and send it along in the request headers of each subsequent request. The easiest way to do this is to use a WebClient for spanning multiple requests, and a CookieContainer to track the cookies for you.

If you're ever unsure about how to mimic the traffic that goes between your browser and a website, a great tool to use is Fiddler. It captures the raw request/response for you to observe.