Getting correct download file size from url

Question

I am trying to get the file size from url (https://windows.php.net/downloads/releases/php-7.2.9-nts-Win32-VC15-x64.zip) and here is the code I have -

HttpWebRequest request = HttpWebRequest.CreateHttp(url);
HttpWebResponse response = (HttpWebResponse)(await request.GetResponseAsync());
long length = response.ContentLength;

But the value of length is 598 bytes whereas the website (and when downloading from browser) reports the size as 24.5MB. I even tried accessing the "Content-Length" from the response header, but it also had the same value, 598.

Am I missing something? Is there any other way to get the file sizes more accurately?

You can check this post: [file size from url](https://stackoverflow.com/questions/122853/how-to-get-the-file-size-from-http-headers) — omriman12, Aug 26 '18 at 15:23
@omriman12, the post you linked talks about the 2 methods which I have already used, response.ContentLength and "Content-Length" in the response header. And they both return incorrect length in this specific case. — ac-lap, Aug 26 '18 at 15:38
Have you looked at what the actual result is? Is it a redirect to elsewhere by any chance, when the result size would be quite understandable? — Sami Kuhmonen, Aug 26 '18 at 15:40
Did you check content Type property of response object...If this is a redirect page it will show you "text" and will return size of that page (which i think is the case here) while for zip file it will show "zip" — A.Learn, Aug 26 '18 at 16:18
@A.Learn, yes the ContentType of the response object is "text/plain" and the StatusCode is OK. I tried making requests with AllowAutoRedirect to true and false, but the response object is same. So, how do I get the final download url? — ac-lap, Aug 26 '18 at 17:07

score 5 · Accepted Answer · edited Jun 20 '20 at 09:12

I used your example URL and read the contents via:

var dataStream = response.GetResponseStream();
var reader = new StreamReader(dataStream);
var responseFromServer = reader.ReadToEnd();

The result I get is:

20/feb/2018: Hi! We seem to be receiving high volume requests coming from empty user agents. While this shouldn't be an issue, this unfortunately resulted in bandwidth issues on this server causing all downloads to be unavailable. We've therefore decided to temporarily block empty user agents until we upgraded our server bandwidth.

03/mar/2018: We've upgraded the server bandwidth. This is however still not sufficient to handle all empty user agent connections. Please update the user agent in your scripts accordingly or contact us so we can discuss it.

Thank you for your understanding.

It says that set UserAgent. So, I set the user agent as follows:

var request = HttpWebRequest.CreateHttp(url);
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1";
var response = (HttpWebResponse)(await request.GetResponseAsync());
var length = response.ContentLength;

Now, I get the correct Content-Length value of 25691309

I just picked an user agent string from: http://www.useragentstring.com/index.php?id=19879

If you just interested in the size of the remote file, you should consider the answer to the linked question. It essentially uses a different HTTP Metho (HEAD vs GET)

        var request = HttpWebRequest.CreateHttp(url);
        request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1";
        request.Method = "HEAD";
        using (var response = await request.GetResponseAsync())
        {
            var length = response.ContentLength;
        }

You are find more details about HEAD Vs GET in the related question: http HEAD vs GET performance

Thanks for the answer. I am writing file download tool, till now I was not setting UserAgent, should I start setting it for all downloads? What are your thoughts on this. — ac-lap, Aug 26 '18 at 17:47
You should have a unique UA string; don't pretend to be a browser since you won't be able to handle HTML, JS, etc. See [this question](https://stackoverflow.com/questions/2601372/what-is-the-standard-format-for-a-browsers-user-agent-string) for more info. — Peter Torr - MSFT, Aug 26 '18 at 18:06

Getting correct download file size from url

1 Answers1