C#: HttpWebRequest POST data not working

Question

I am developing a C# wpf application that has a functionality of logging into my website and download the file. This said website has an Authorize attribute on its action. I need 2 cookies for me to able to download the file, first cookie is for me to log in, second cookie(which is provided after successful log in) is for me to download the file. So i came up with the flow of keeping my cookies after my httpwebrequest/httpwebresponse. I am looking at my posting flow as maybe it is the problem. Here is my code.

void externalloginanddownload()
{
    string pageSource = string.Empty;
    CookieContainer cookies = new CookieContainer();

    HttpWebRequest getrequest = (HttpWebRequest)WebRequest.Create("login uri");
    getrequest.CookieContainer = cookies;
    getrequest.Method = "GET";
    getrequest.AllowAutoRedirect = false;

    HttpWebResponse getresponse = (HttpWebResponse)getrequest.GetResponse();

    using (StreamReader sr = new StreamReader(getresponse.GetResponseStream()))
    {
        pageSource = sr.ReadToEnd();
    }

    var values = new NameValueCollection
    {
                {"Username", "username"},
                {"Password", "password"},
        { "Remember me?","False"},
            };

    var parameters = new StringBuilder();

    foreach (string key in values.Keys)
    {
        parameters.AppendFormat("{0}={1}&",
            HttpUtility.UrlEncode(key),
            HttpUtility.UrlEncode(values[key]));
    }

    parameters.Length -= 1;

    HttpWebRequest postrequest = (HttpWebRequest)WebRequest.Create("login uri");
    postrequest.CookieContainer = cookies;
    postrequest.Method = "POST";

    using (var writer = new StreamWriter(postrequest.GetRequestStream()))
    {
        writer.Write(parameters.ToString());
    }

    using (WebResponse response = postrequest.GetResponse()) // the error 500 occurs here
    {
        using (var streamReader = new StreamReader(response.GetResponseStream()))
        {
            string html = streamReader.ReadToEnd();

        }
    }
}

you haven't mentioned what problem you are having. Where does it fail? — Garr Godfrey, Jul 19 '17 at 05:42
it fails gettting the response of the httpwebrequest with a POST method. it gives me an ERROR 500. — White Hat, Jul 19 '17 at 05:42
i just saw it on one of the SO post. and to figure it out, it is just removing the '&' sign of the last parameter — White Hat, Jul 19 '17 at 05:45
can someone give me another way to download a file from a website which has an [Authorize] attribute? WebClient doesnt seems to work for me. Thanks — White Hat, Jul 19 '17 at 05:47
I think the parameters which you are sending is not in proper format. — Ravikumar, Jul 19 '17 at 05:51
[HttpPost] [AllowAnonymous] [ValidateAntiForgeryToken] public async Task Login(UsernamePwdModel model, string returnUrl) {} — White Hat, Jul 19 '17 at 05:52
I am passing that to this action. and the viewmodel has username[string], password[string] and remember me[bool] — White Hat, Jul 19 '17 at 05:53
also, check the enctype on the
tag on login page, and make sure you are using the correct element name attributes. "Remember me?" is technically a legal name, but seems very unlikely that was used as the name attribute. — Garr Godfrey, Jul 19 '17 at 06:37

Garr Godfrey · Answer 1 · 2017-07-19T06:19:16.783

1

When you get the WebResponse, the cookies returned will be in the response, not in the request (oddly enough, even though you need to CookieContainer on the request).

You will need to add the cookies from the response object to your CookieContainer, so it gets sent on the next request.

One simple way:

for(var cookie in getresponse.Cookies) 
    cookies.Add(cookie)

Since the cookies in response is already a cookies container, you can do this (might help to check for null in case all cookies were already there)

if (response.Cookies != null) cookies.Add(response.Cookies)

You may also have trouble with your POST as you need to set ContentType and length:

myWebRequest.ContentLength = parameters.Length;
myWebRequest.AllowWriteStreamBuffering = true;

If you have any multibyte characters to think about, you may have to address that as well by setting the encoding to UTF-8 on the request and the stringbuilder, and converting string to bytes and using that length.

Another tip: some web server code chokes if there is no user agent. Try:

myWebRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)";

And just in case you have any multibyte characters, it is better to do this:

var databytes = System.Text.Encoding.UTF8.GetBytes(parameters.ToString());
myWebRequest.ContentLength = databytes.Length;
myWebRequest.ContentType = "application/x-www-form-urlencoded; charset=utf-8";
using (var stream = myWebRequest.GetRequestStream())
{
    stream.Write(databytes, 0, databytes.Length);
}

edited Jul 19 '17 at 06:19

answered Jul 19 '17 at 05:53

Garr Godfrey

8,257
2
25
23

The response *should* only include new cookies that weren't in the request, so doing this after each request should accomplish what you want – Garr Godfrey Jul 19 '17 at 05:55
im not sure about the "application/x-www-form-urlencoded" does it mean the values that i am passing to the action is placed on the URL? – White Hat Jul 19 '17 at 06:06
"application/x-www-form-urlencoded" basically means that the post parameters are formatted as "key=value&key=value..." as opposed to json encoding or some other type. – Garr Godfrey Jul 19 '17 at 06:08
more detail: https://stackoverflow.com/questions/4007969/application-x-www-form-urlencoded-or-multipart-form-data – Garr Godfrey Jul 19 '17 at 06:09
I am still getting this error. The remote server returned an error: (500) Internal Server Error. – White Hat Jul 19 '17 at 06:10
(500) is coming from your web server. It is choking on the input for some reason, so either a bug in the server, or something isn't passed right. Try a debugger. – Garr Godfrey Jul 19 '17 at 06:14
Is there any way to download a file that requires login before you can download it? i changed the login uri to the actual download uri and i got a response of the actual html code of my log in page. And my suspect is maybe i cant successfully logged in. – White Hat Jul 19 '17 at 06:20
"login" isn't standard, so no built in way. If the website used basic authentication, that would be different, but if you created the website you'd know best how it worked. – Garr Godfrey Jul 19 '17 at 06:29
Yes that is absolutely right. I would need to have an integration with the creator of the website. Thank you for the response Garr Godfrey! – White Hat Jul 19 '17 at 06:34

score 0 · Answer 2 · answered Jul 19 '17 at 06:57

In C# Application (Server side Web API) Enable the C++ Exception and Common Language Run time Exceptions using (Ctrl+Alt+E) what is the Server side Exception it's throw.

First you check data is binding Properly. After you can see what it is Exact Exception. the Internal Server Error Mostly throw the data is not correct format and not properly managed Exception.

C#: HttpWebRequest POST data not working

2 Answers2