Polly policy not working using "AddPolicyHandler"

Question

I have an application that makes a request for an authenticated service, where it is necessary to pass the access_token.

My idea is to use Polly to retry if the access_token is expired.

I'm using Refit (v5.1.67) and Polly (v7.2.1) in a .NET Core 3.1 application.

The services are registered as follows:

services.AddTransient<ExampleDelegatingHandler>();

IAsyncPolicy<HttpResponseMessage> retryPolicy = Policy<HttpResponseMessage>
    .Handle<ApiException>()
    .RetryAsync(1, (response, retryCount) =>
    {
        System.Diagnostics.Debug.WriteLine($"Polly Retry => Count: {retryCount}");
    });

services.AddRefitClient<TwitterApi>()
    .ConfigureHttpClient(c =>
    {
        c.BaseAddress = new Uri("https://api.twitter.com/");
    })
    .AddHttpMessageHandler<ExampleDelegatingHandler>()
    .AddPolicyHandler((sp, req) =>
    {
        //this policy does not works, because the exception is not catched on 
        //"Microsoft.Extensions.Http.PolicyHttpMessageHandler" (DelegatingHandler)
        return retryPolicy;
    });

public interface TwitterApi
{
    [Get("/2/users")]
    Task<string> GetUsers();
}

public class ExampleDelegatingHandler : DelegatingHandler
{
    protected async override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        try
        {
            return await base.SendAsync(request, cancellationToken);
        }
        catch (Exception)
        {
            //Why do not catch the exception?
            throw;
        }
    }
}

The retry policy is not working!

Analyzing the problem, I realized that the exception is not being caught inside the HttpClient's DelegatingHandler. Since the AddPolicyHandler statement is generating a DelegatingHandler (PolicyHttpMessageHandler) to execute the policy and the exception is not caught there, the policy never executes. I realized that the problem only occurs in asynchronous scenarios, where the request can be sent. In synchronous scenarios it works (example: timeout).

Why the exception is not caught inside DelegatingHandler??

I am attaching an example project simulating a Twitter call.

https://www.dropbox.com/s/q1797rq1pbjvcls/ConsoleApp2.zip?dl=0

External references:

https://github.com/reactiveui/refit#using-httpclientfactory

https://www.hanselman.com/blog/UsingASPNETCore21sHttpClientFactoryWithRefitsRESTLibrary.aspx

https://learn.microsoft.com/en-us/aspnet/core/fundamentals/http-requests?view=aspnetcore-3.1

Is there any particular reason why did you register a custom `MessageHandler`? Also you can simplify the `AddPolicyHandler` call by passing the policy itself : `.AddPolicyHandler(retryPolicy);` — Peter Csala, Aug 25 '20 at 14:15
Peter, this is what I want (to use AddPolicyHandler). I registered a custom MessageHandle only for didactic purpose. AddPolicyHandler behide the scenes register a message handle (PolicyHttpMessageHandler). My point is that the exception is not caught there, so the policy is never executed. — Marcelo Diniz, Sep 01 '20 at 22:38

score 37 · Answer 1 · answered Nov 16 '21 at 16:53

37

I had an issue involving .NET 5 >= with Polly and HttpClient, which the compiler showed: HttpClientBuilder does not contain a definition for AddPolicyHandler. I could fix it when I changed the Nuget PackagePolly.Extensions.Http to Microsoft.Extensions.Http.Polly, I know that isn't the same situation reported here but it might be useful for other people who have come here to find this answer, like me.

answered Nov 16 '21 at 16:53

Anderson Paiva

761
9
14

4

Very useful indeed! Exactly the issue I was having, thanks! – Beuz Nov 19 '21 at 18:33
thanks a lot this helped me as well u da man – Alex Gordon Oct 26 '22 at 17:11

score 12 · Answer 2 · answered Sep 02 '20 at 07:38

TL;DR: The ordering of AddPolicyHandler and AddHttpMessageHandler does matter.

I've recreated the problem with HttpClient (so without Refit).

Typed HttpClient for testing

public interface ITestClient
{
    Task<string> Get();
}

public class TestClient: ITestClient
{
    private readonly HttpClient client;
    public TestClient(HttpClient client)
    {
        this.client = client;
    }
    public async Task<string> Get()
    {
        var resp = await client.GetAsync("http://not-existing.site");
        return "Finished";
    }
}

Controller for testing

[ApiController]
[Route("[controller]")]
public class TestController : ControllerBase
{
    private readonly ITestClient client;

    public TestController(ITestClient client)
    {
        this.client = client;
    }

    [HttpGet]
    public async Task<string> Get()
    {
        return await client.Get();
    }
}

DelegateHandler for testing

public class TestHandler: DelegatingHandler
{
    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        try
        {
            return await base.SendAsync(request, cancellationToken);
        }
        catch (System.Exception ex)
        {
            _ = ex;
            throw;
        }
    }
}

Ordering #1 - Handler, Policy

Startup

public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers();
    services.AddTransient<TestHandler>();
    services.AddHttpClient<ITestClient, TestClient>()
        .AddHttpMessageHandler<TestHandler>() //Handler first
        .AddPolicyHandler(RetryPolicy()); //Policy second
}

private IAsyncPolicy<HttpResponseMessage> RetryPolicy()
    => Policy<HttpResponseMessage>
    .Handle<HttpRequestException>()
    .RetryAsync(1, (resp, count) =>
    {
        Console.WriteLine(resp.Exception);
    });

Execution order

TestController's Get
TestClient's Get
TestHandler's SendAsync's try
RetryPolicy's onRetry
TestHandler's SendAsync's catch
TestController's Get fails with HttpRequestException (inner: SocketException)

So, here the retry policy does not fired.

Ordering #2 - Policy, Handler

Startup

public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers();
    services.AddTransient<TestHandler>();
    services.AddHttpClient<ITestClient, TestClient>()
        .AddPolicyHandler(RetryPolicy()) //Policy first
        .AddHttpMessageHandler<TestHandler>(); //Handler second
}

private IAsyncPolicy<HttpResponseMessage> RetryPolicy()
    => Policy<HttpResponseMessage>
    .Handle<HttpRequestException>()
    .RetryAsync(1, (resp, count) =>
    {
        Console.WriteLine(resp.Exception);
    });

Execution order

TestController's Get
TestClient's Get
TestHandler's SendAsync's try
TestHandler's SendAsync's catch
RetryPolicy's onRetry
TestHandler's SendAsync's try
TestHandler's SendAsync's catch
TestController's Get fails with HttpRequestException (inner: SocketException)

So, here the retry policy has been fired.

score 4 · Answer 3 · answered Jul 01 '22 at 13:31

1. Why

At the time policies and delegating handlers are executed, a failed HTTP response is not an exception yet. It's just an instance of HttpResponseMessage with an unsuccessful status. Refit converts this status into an exception as the very last step in the request-response processing.

2. Order

As correctly noted in Peter Csala's answer, order matters. When a request is made:

Refit serializes the parameters into an HttpRequestMessage and passes it to HttpClient
HttpClient does initial preparations
HttpClient runs the request messages through the handlers and policies in the order they were added to the client
The resulting message is sent to the server
Server's response is converted to a HttpResponseMessage object
This object bubbles up through the same sequence of handlers and policies but in reverse order
HttpClient does final processing and returns the result to Refit
Refit converts any errors into ApiExceptions

Therefore a retry policy will re-run everything that was added after it, but whatever was before it will be executed only once.

So if you want your access_token to be re-generated on every retry, the delegating handler that creates the token must be registered after the retry policy.

3. How

The easiest way to retry on HTTP failures is to use HttpPolicyExtensions.HandleTransientHttpError() from Polly.Extensions.Http. Otherwise you'd have to check for all the failure HTTP status codes yourself. The benefit of HandleTransientHttpError is that it only retries on failures which makes sense to retry, like 500 or socket errors. On the other hand it will not retry a 404 for instance, because the resource is not there and is unlikely to reappear if we try again.

This was exactly what I was looking for and helped me to solve the issue I was running into. I was trying to handle the `ApiException`, not realizing this was too late to the game. Thanks! — Emiel Koning, Dec 23 '22 at 09:55

score 2 · Answer 4 · answered May 13 '22 at 14:53

I think if we change the policy

IAsyncPolicy<HttpResponseMessage> retryPolicy = Policy<HttpResponseMessage>
    .Handle<ApiException>()
    .RetryAsync(1, (response, retryCount) =>
    {
        System.Diagnostics.Debug.WriteLine($"Polly Retry => Count: {retryCount}");
    });

to

.HandleResult(x => !x.IsSuccessStatusCode)

or

.HandleResult(x => _retryStatusCodes.Contains(x.StatusCode))
...
private static readonly ISet<HttpStatusCode> _retryStatusCodes = new HashSet<HttpStatusCode>
        {
            HttpStatusCode.RequestTimeout,
            HttpStatusCode.BadGateway,
            HttpStatusCode.ServiceUnavailable,
            HttpStatusCode.GatewayTimeout,
        };

then it should work.

IAsyncPolicy<HttpResponseMessage> retryPolicy = Policy<HttpResponseMessage>

.HandleResult(x => _retryStatusCodes.Contains(x.StatusCode))
.RetryAsync(1, (response, retryCount) =>
{
    System.Diagnostics.Debug.WriteLine($"Polly Retry => Count: {retryCount}");
});

Maybe Refit checks status code and throws ApiException on some later stage

Polly policy not working using "AddPolicyHandler"

4 Answers4

Typed HttpClient for testing

Controller for testing

DelegateHandler for testing

Ordering #1 - Handler, Policy

Startup

Execution order

Ordering #2 - Policy, Handler

Startup

Execution order

1. Why

2. Order

3. How

Linked