How can I download a specific CSS file in C#?

Question

I'm using WebClient and I'm trying to download this CSS file: https://assets.staples-static.com/NC1/pn/bui/20181010140727/styleguide/docs/css/atcOverlay.css

This is the code I tried using:

WebClient webClient = new WebClient();
string css = webClient.DownloadString("https://assets.staples-static.com/NC1/pn/bui/20181010140727/styleguide/docs/css/atcOverlay.css");

It didn't work - I got a timeout exception.

Then I tried adding the same headers as the browser:

WebClient webClient = new WebClient();
webClient.Headers.Add(HttpRequestHeader.Pragma, "no-cache");
webClient.Headers.Add(HttpRequestHeader.CacheControl, "no-cache");
webClient.Headers.Add("Upgrade-Insecure-Requests", "1");
webClient.Headers.Add(HttpRequestHeader.UserAgent, "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36");
webClient.Headers.Add(HttpRequestHeader.Accept, "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8");
webClient.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip, deflate, br");
webClient.Headers.Add(HttpRequestHeader.AcceptLanguage, "en-US,en;q=0.9");
string css = webClient.DownloadString("https://assets.staples-static.com/NC1/pn/bui/20181010140727/styleguide/docs/css/atcOverlay.css");

Same result. I even tried replacing webClient.DownloadString with webClient.DownloadData, and even replace it completely with HttpWebRequest. All gave me the same results.

The only differences I see when I use Fiddler (I look at the Raw tab) are the casing of the Connection header (Keep-Alive in C# vs. keep-alive in browsers), and the order of the headers (which I tried to match as much as I could, but I just can't find a way to put the HOST and CONNECTION first below the GET).

Some other CSS files I try to download from the same host and from other hosts download just fine.

Did anyone encounter such issue? Am I missing anything? What else can I try?

Update:

I tried Poul Bak's suggestion from comments below. It didn't work. This is the code I used:

HttpClient httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Clear();
httpClient.DefaultRequestHeaders.Add("Host", "assets.staples-static.com");
httpClient.DefaultRequestHeaders.Add("Pragma", "no-cache");
httpClient.DefaultRequestHeaders.Add("Cache-Control", "no-cache");
httpClient.DefaultRequestHeaders.Add("Upgrade-Insecure-Requests", "1");
httpClient.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36");
httpClient.DefaultRequestHeaders.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8");
httpClient.DefaultRequestHeaders.Add("Accept-Encoding", "gzip, deflate, br");
httpClient.DefaultRequestHeaders.Add("Accept-Language", "en-US,en;q=0.9");
var result = httpClient.GetAsync("https://assets.staples-static.com/NC1/pn/bui/20181010140727/styleguide/docs/css/atcOverlay.css").Result;

Try using 'HttpClient', it's my experience, that it Works much better. (I can't see anything wrong with your code). — Poul Bak, Oct 18 '18 at 14:10
Have you tried using 'http' instead of 'https' ? It still Works in browser, so... — Poul Bak, Oct 18 '18 at 14:13
@PoulBak Thank you, I tried that. Unfortunately, same results. I'll update the question to show the code I used. — Itai Bar-Haim, Oct 18 '18 at 14:34
@PoulBak regarding using `HTTP` instead of `HTTPS` - unfortunately I don't provide the URIs and I don't want to change them in case the server I connect to won't be able to provide them. — Itai Bar-Haim, Oct 18 '18 at 14:40
Might be related: https://stackoverflow.com/questions/52498248/unable-to-read-data-from-the-transport-connectionan-existing-connection-was-for — Lennart Stoop, Oct 18 '18 at 14:42

score 1 · Accepted Answer · answered Oct 18 '18 at 15:09

I did some tests an found a way to make it Work. The problem is the 'Connection' header value not being set to 'Keep-Alive'.

So setting that makes it Work:

    HttpClient httpClient = new HttpClient();
    httpClient.DefaultRequestHeaders.Clear();
    httpClient.DefaultRequestHeaders.Add("Connection", "Keep-Alive");
    httpClient.DefaultRequestHeaders.Add("Host", "assets.staples-static.com");
    httpClient.DefaultRequestHeaders.Add("Pragma", "no-cache");
    httpClient.DefaultRequestHeaders.Add("Cache-Control", "no-cache");
    httpClient.DefaultRequestHeaders.Add("Upgrade-Insecure-Requests", "1");
    httpClient.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36");
    httpClient.DefaultRequestHeaders.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8");
    httpClient.DefaultRequestHeaders.Add("Accept-Encoding", "gzip, deflate, br");
    httpClient.DefaultRequestHeaders.Add("Accept-Language", "en-US,en;q=0.9");
    var result = await httpClient.GetAsync("https://assets.staples-static.com/NC1/pn/bui/20181010140727/styleguide/docs/css/atcOverlay.css");
    string text = await result.Content.ReadAsStringAsync();

Now I get about 10kb string.

What about .net 3.5 and 4.0? It seems `HttpClient` is missing in those frameworks, and `WebClient` still doesn't work. Any replacement? — Itai Bar-Haim, Oct 21 '18 at 07:02
HttpCient is the replacement for WebClient. May be be you can use HttpWebRequest . — Poul Bak, Oct 21 '18 at 12:44

How can I download a specific CSS file in C#?

Update:

1 Answers1