Download a file but it seems like I have to do it with a browser

Question

I would like to download a file from a website but the specific URL does not work unless a different page is loaded.

I literally call a web browser to load a page and then call another page to download a file.

This works but I hate it...

static void Main(string[] args)
        {
            //Open website
            Process.Start("chrome.exe", "https://www.fapiis.gov/fapiis/allfapiisdata.action");

            //Wait 20 sec
            System.Threading.Thread.Sleep(20000);

            //DownLoad File
            Process.Start("Chrome.exe", "https://www.fapiis.gov/fapiis/downloadview?type=allFapiis");

            //Wait 10 sec
            System.Threading.Thread.Sleep(10000);

            //Close Chrome
            Process[] chromeInstances = Process.GetProcessesByName("chrome");
            foreach (Process p in chromeInstances)
            { p.Kill(); }

            //Move file for SSIS Processing
            string[] files = System.IO.Directory.GetFiles(@"C:\Users\xxxx\Downloads", "AllFapiis*");
            foreach (string file in files)
            {
                string fname = System.IO.Path.GetFileName(file);
                System.IO.File.Copy(file, @"C:\xxxxx\FAPIIS\" + fname);
                System.IO.File.Delete(file);
            }
        }

I tried using webclient but it can never find the 2nd page. Any other way to do this without calling a browser? Another problem is I can't direct where the download goes. It automatically goes into downloads on my user account.

score 3 · Accepted Answer · answered Sep 25 '19 at 16:49

The site you are trying to download from, uses cookies. So once you navigate to the first page, some cookies are set to the browser. This allows the download on the second page.

You could accomplish this with this code:

static async Task TryDownload()
{
    var clientHandler = new HttpClientHandler
    {
        AllowAutoRedirect = true,
        UseCookies = true,
        CookieContainer = new CookieContainer()
    };

    using (var httpClient = new HttpClient(clientHandler))
    {
        // this gets the request and allows the site to set cookies.
        var warmup = await httpClient.GetAsync("https://www.fapiis.gov/fapiis/allfapiisdata.action");

        // get the file (cookies are sent automatically).
        var fileResponse = httpClient.GetAsync("https://www.fapiis.gov/fapiis/downloadview?type=allFapiis");

        if (fileResponse.Result.IsSuccessStatusCode)
        {
            HttpContent content = fileResponse.Result.Content;
            var contentStream = await content.ReadAsStreamAsync();
            using (var fileStream = File.Create("C:\\temp\\a.xlsx"))
            {
                contentStream.CopyTo(fileStream);
            }
        }
    }
}

Output:

Excel file in c:\temp (662,044 bytes)

In this implementation, it not returning anything. You can change the signature to Task or any other generic type to return some value. — EylM, Sep 25 '19 at 18:15
I am calling from main with var a = TryDownload(); the code exits after just trying to do warmup. Am I doing something wrong? — KeithL, Sep 25 '19 at 18:44

Download a file but it seems like I have to do it with a browser

1 Answers1

Linked