How to use HttpWebRequest (.NET) asynchronously?

Question

How can I use HttpWebRequest (.NET, C#) asynchronously?

Answer 1

Use HttpWebRequest.BeginGetResponse()

HttpWebRequest webRequest;

void StartWebRequest()
{
    webRequest.BeginGetResponse(new AsyncCallback(FinishWebRequest), null);
}

void FinishWebRequest(IAsyncResult result)
{
    webRequest.EndGetResponse(result);
}

The callback function is called when the asynchronous operation is complete. You need to at least call EndGetResponse() from this function.

Answer 2

By far the easiest way is by using TaskFactory.FromAsync from the TPL. It's literally a couple of lines of code when used in conjunction with the new async/await keywords:

var request = WebRequest.Create("http://www.stackoverflow.com");
var response = (HttpWebResponse) await Task.Factory
    .FromAsync<WebResponse>(request.BeginGetResponse,
                            request.EndGetResponse,
                            null);
Debug.Assert(response.StatusCode == HttpStatusCode.OK);

If you can't use the C#5 compiler then the above can be accomplished using the Task.ContinueWith method:

Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse,
                                    request.EndGetResponse,
                                    null)
    .ContinueWith(task =>
    {
        var response = (HttpWebResponse) task.Result;
        Debug.Assert(response.StatusCode == HttpStatusCode.OK);
    });

Answer 3

Considering the answer:

HttpWebRequest webRequest;

void StartWebRequest()
{
    webRequest.BeginGetResponse(new AsyncCallback(FinishWebRequest), null);
}

void FinishWebRequest(IAsyncResult result)
{
    webRequest.EndGetResponse(result);
}

You could send the request pointer or any other object like this:

void StartWebRequest()
{
    HttpWebRequest webRequest = ...;
    webRequest.BeginGetResponse(new AsyncCallback(FinishWebRequest), webRequest);
}

void FinishWebRequest(IAsyncResult result)
{
    HttpWebResponse response = (result.AsyncState as HttpWebRequest).EndGetResponse(result) as HttpWebResponse;
}

Greetings

Answer 4

Everyone so far has been wrong, because BeginGetResponse() does some work on the current thread. From the documentation:

The BeginGetResponse method requires some synchronous setup tasks to complete (DNS resolution, proxy detection, and TCP socket connection, for example) before this method becomes asynchronous. As a result, this method should never be called on a user interface (UI) thread because it might take considerable time (up to several minutes depending on network settings) to complete the initial synchronous setup tasks before an exception for an error is thrown or the method succeeds.

So to do this right:

void DoWithResponse(HttpWebRequest request, Action<HttpWebResponse> responseAction)
{
    Action wrapperAction = () =>
    {
        request.BeginGetResponse(new AsyncCallback((iar) =>
        {
            var response = (HttpWebResponse)((HttpWebRequest)iar.AsyncState).EndGetResponse(iar);
            responseAction(response);
        }), request);
    };
    wrapperAction.BeginInvoke(new AsyncCallback((iar) =>
    {
        var action = (Action)iar.AsyncState;
        action.EndInvoke(iar);
    }), wrapperAction);
}

You can then do what you need to with the response. For example:

HttpWebRequest request;
// init your request...then:
DoWithResponse(request, (response) => {
    var body = new StreamReader(response.GetResponseStream()).ReadToEnd();
    Console.Write(body);
});

Answer 5

public static async Task<byte[]> GetBytesAsync(string url) {
    var request = (HttpWebRequest)WebRequest.Create(url);
    using (var response = await request.GetResponseAsync())
    using (var content = new MemoryStream())
    using (var responseStream = response.GetResponseStream()) {
        await responseStream.CopyToAsync(content);
        return content.ToArray();
    }
}

public static async Task<string> GetStringAsync(string url) {
    var bytes = await GetBytesAsync(url);
    return Encoding.UTF8.GetString(bytes, 0, bytes.Length);
}

Answer 6

I ended up using BackgroundWorker, it is definitely asynchronous unlike some of the above solutions, it handles returning to the GUI thread for you, and it is very easy to understand.

It is also very easy to handle exceptions, as they end up in the RunWorkerCompleted method, but make sure you read this: Unhandled exceptions in BackgroundWorker

I used WebClient but obviously you could use HttpWebRequest.GetResponse if you wanted.

var worker = new BackgroundWorker();

worker.DoWork += (sender, args) => {
    args.Result = new WebClient().DownloadString(settings.test_url);
};

worker.RunWorkerCompleted += (sender, e) => {
    if (e.Error != null) {
        connectivityLabel.Text = "Error: " + e.Error.Message;
    } else {
        connectivityLabel.Text = "Connectivity OK";
        Log.d("result:" + e.Result);
    }
};

connectivityLabel.Text = "Testing Connectivity";
worker.RunWorkerAsync();

Answer 7

.NET has changed since many of these answers were posted, and I'd like to provide a more up-to-date answer. Use an async method to start a Task that will run on a background thread:

private async Task<String> MakeRequestAsync(String url)
{    
    String responseText = await Task.Run(() =>
    {
        try
        {
            HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
            WebResponse response = request.GetResponse();            
            Stream responseStream = response.GetResponseStream();
            return new StreamReader(responseStream).ReadToEnd();            
        }
        catch (Exception e)
        {
            Console.WriteLine("Error: " + e.Message);
        }
        return null;
    });

    return responseText;
}

To use the async method:

String response = await MakeRequestAsync("http://example.com/");

Update:

This solution does not work for UWP apps which use WebRequest.GetResponseAsync() instead of WebRequest.GetResponse(), and it does not call the Dispose() methods where appropriate. @dragansr has a good alternative solution that addresses these issues.

Answer 8

public void GetResponseAsync (HttpWebRequest request, Action<HttpWebResponse> gotResponse)
    {
        if (request != null) { 
            request.BeginGetRequestStream ((r) => {
                try { // there's a try/catch here because execution path is different from invokation one, exception here may cause a crash
                    HttpWebResponse response = request.EndGetResponse (r);
                    if (gotResponse != null) 
                        gotResponse (response);
                } catch (Exception x) {
                    Console.WriteLine ("Unable to get response for '" + request.RequestUri + "' Err: " + x);
                }
            }, null);
        } 
    }

Answer 9

Follow up to the @Isak 's answer, which is very good. Nonetheless it's biggest flaw is that it will only call the responseAction if the response has status 200-299. The best way to fix this is:

private void DoWithResponseAsync(HttpWebRequest request, Action<HttpWebResponse> responseAction)
{
    Action wrapperAction = () =>
    {
        request.BeginGetResponse(new AsyncCallback((iar) =>
        {
            HttpWebResponse response;
            try
            {
                response = (HttpWebResponse)((HttpWebRequest)iar.AsyncState).EndGetResponse(iar);
            }
            catch (WebException ex)
            {
                // It needs to be done like this in order to read responses with error status:
                response = ex.Response as HttpWebResponse;
            }
            responseAction(response);
        }), request);
    };
    wrapperAction.BeginInvoke(new AsyncCallback((iar) =>
    {
        var action = (Action)iar.AsyncState;
        action.EndInvoke(iar);
    }), wrapperAction);
}

And then as @Isak follows:

HttpWebRequest request;
// init your request...then:
DoWithResponse(request, (response) => {
    var body = new StreamReader(response.GetResponseStream()).ReadToEnd();
    Console.Write(body);
});

Answer 10

I've been using this for async UWR, hopefully it helps someone

    string uri = "http://some.place.online";

    using (UnityWebRequest uwr = UnityWebRequest.Get(uri))
    {
        var asyncOp = uwr.SendWebRequest();
        while (asyncOp.isDone == false) await Task.Delay(1000 / 30); // 30 hertz

        if(uwr.result == UnityWebRequest.Result.Success) return uwr.downloadHandler.text;
        Debug.LogError(uwr.error);
    }