How to Retrieve file from URL which Requires Authentication?

Question

I'm making a tool in Unity to retrieve data from a server. The server's interface can provide URLs that we can later click on which will return an XML or CSV file with the results of that query from that server. But, it requires Basic Authentication. When clicking the links, it simply pops up a login screen before giving me the results. If I try what I [think] I know in Unity (starting with WebRequest.GetResponse()) it simply fails and says I am not authorized. It does not show the popup for authentication. So how do I let that login popup appear when accessing with Unity and await the login results to get the file? Or is there some standardized way to provide that info in the link itself?

You need to use something called authentication headers. I think this is a good explenation: https://stackoverflow.com/questions/4334521/httpwebrequest-using-basic-authentication — Tomer Shahar, Feb 06 '20 at 14:08
What kind of authentication that you are talking about? cause there are multiple Auth methods that can be used for API authentication. First you have figure that out and after that, you can add whatever is it in your headers(generally API require auth send Authentication related parameters in headers, though some API also uses API key as user query). — Ghost The Punisher, Feb 06 '20 at 14:23
if it is possible can I have an API link if you don't mind so that I can study? — Ghost The Punisher, Feb 06 '20 at 14:24
It seems to be basic authentication. I'm attempting with some of the stuff via the link above, but there appears to be so many different ways/things to try. We'll see if it gets anywhere! — CodeMonkey, Feb 06 '20 at 14:36
Egads! Thanks @TomerShahar! This answer https://stackoverflow.com/a/13956730/1411541 got me the response and this helped me save the response to file! http://www.java2s.com/Tutorial/CSharp/0580__Network/SavewhatyoureadfromWebRequesttoafile.htm — CodeMonkey, Feb 06 '20 at 14:45
**Rather checkout this!** https://stackoverflow.com/questions/39482954/unitywebrequest-embedding-user-password-data-for-http-basic-authentication-not which shows how to do it for a `UnityWebRequest` — derHugo, Feb 06 '20 at 17:21

score 0 · Answer 1 · edited Feb 06 '20 at 15:12

Here is some code that should you get started. Just fill in the request link and username, password. please see the comments in the code to see what it does.

//try just in case something went wrong whith calling the api
try
{
    //Use using so that if the code end the client disposes it self
    using (HttpClient client = new HttpClient())
    {
        //Setup authentication information
        string yourusername = "username";
        string yourpwd = "password";
        //this is when you expect json to return from the api
        client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
        //add the authentication to the request
        client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", 
            Convert.ToBase64String(
                System.Text.ASCIIEncoding.ASCII.GetBytes($"{yourusername}:{yourpwd}")));
        //api link used to make the call
        var requestLink = $"apiLink";
        using (HttpResponseMessage response = client.GetAsync(requestLink).Result)
        {
            //Make sure the request was successfull before proceding
            response.EnsureSuccessStatusCode();
            //Get response from website and convert to a string
            string responseBody = response.Content.ReadAsStringAsync().Result;
            //now you have the results
        }
    }
}
//Catch the exception if something went from and show it!
catch (Exception)
{
    throw;
}

Urgh .. you should really rather use `UnityWebRequest.Get` for this... Can use the header part the same way there : https://stackoverflow.com/questions/39482954/unitywebrequest-embedding-user-password-data-for-http-basic-authentication-not — derHugo, Feb 06 '20 at 17:16

score 0 · Accepted Answer · answered Feb 10 '20 at 17:58

This is what I ended up going with after looking at the comments above. Let me know if I'm doing anything terribly inefficient!

    String username = "Superman"; // Obviously handled secretly
    String pw = "ILoveLex4evar!"; // Obviously handled secretly
    String url = "https://www.SuperSecretServer.com/123&stuff=?uhh";
    String encoded = System.Convert.ToBase64String(System.Text.Encoding.GetEncoding("ISO-8859-1").GetBytes(username + ":" + pw));
    CookieContainer myContainer = new CookieContainer();
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    request.Headers.Add("Authorization", "Basic " + encoded);
    try
    {
        using (WebResponse response = request.GetResponse())
        {
            using (Stream responseStream = response.GetResponseStream())
            {
                using (FileStream xml = File.Create("filepath/filename.xml"))
                {
                    byte[] buffer = new byte[BufferSize];
                    int read;
                    while ((read = responseStream.Read(buffer, 0, buffer.Length)) > 0)
                    {
                        xml.Write(buffer, 0, read);
                    }
                }
            }
        }
    }

How to Retrieve file from URL which Requires Authentication?

2 Answers2