HttpWebRequest GetResponse got redirected

Question

Thanks for responses! Sorry I got my question wrong so I am rephrasing my question as below:

I need to read some text values from one of our old aspx pages of one internal website.

When I directly put url in a newly opened browser, first time, it got re-directed to default page, I need to click on a link to get the the page I wanted; then I put in the same url in the same browser window and hit Enter again, it goes to correct page and everything looks good. I think the second time works because there are some session variable established.

How can I achieve this from code? When I use following code trying to read that page (webReq.GetResponse()), I can see from Fiddler that it first hit the page I wanted, then it got redirected to the default.aspx of that site - exactly like if I do it manually first time with new browser window.

Here is the code, I have tried call webReq.GetResponse() second time but Fiddler doesn't show any traffic for my second call. Any suggestions? - Thanks!

string url = "http://mycompany.com/page1.aspx";

HttpWebRequest webReq = (HttpWebRequest)HttpWebRequest.Create(url);
try
{
    webReq.CookieContainer = new CookieContainer();
    webReq.UserAgent = "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko";
    webReq.Method = "GET";
    webReq.Credentials = CredentialCache.DefaultCredentials;

    using (WebResponse response = webReq.GetResponse())
    {
        using (Stream stream = response.GetResponseStream())
        {
            StreamReader reader = new StreamReader(stream);
            string res = reader.ReadToEnd();
            ...
        }
    }
}
catch (Exception ex)
{
    ...
}

Can you take a look at Examples given at ; https://learn.microsoft.com/fr-fr/dotnet/api/system.net.httpwebrequest.getresponse?view=netframework-4.8 or https://learn.microsoft.com/fr-fr/dotnet/framework/network-programming/how-to-request-data-using-the-webrequest-class ?? — Mohamad TAGHLOBI, Jan 08 '20 at 22:22
@Mohamad, thanks, I added missed webReq.Credentials = CredentialCache.DefaultCredentials, but still not working... — startover, Jan 09 '20 at 14:21
From a browser follow the traffic with fiddler, looking especially for any 401 Authentication challenges or Authentication headers in the traffic. — David Browne - Microsoft, Jan 09 '20 at 14:23
After logging in, if you get a `301 Redirect` response code, you will need to capture the redirect URI and follow the redirect to get a `200 OK`. — Cameron Tinker, Jan 09 '20 at 19:06
Hi #startover, can you try please to compress and then decompress your webReq response as show in this link : https://stackoverflow.com/questions/27108264/c-sharp-how-to-properly-make-a-http-web-get-request — Mohamad TAGHLOBI, Jan 10 '20 at 18:26

score 0 · Answer 1 · answered Jan 09 '20 at 20:01

Thanks for responding!

To mimic how it works in browser, I have to use HttpClient instead of HttpWebRequest, and call GetStringAsync twice to get to the page I need to go, as shown here

using (HttpClient client = new HttpClient())
{
     var html = await client.GetStringAsync(url);
     html = await client.GetStringAsync(url);
     ...
}

HttpWebRequest GetResponse got redirected

1 Answers1