Web API 2 return OK response but continue processing in the background

Question

I have create an mvc web api 2 webhook for shopify:

public class ShopifyController : ApiController
{
    // PUT: api/Afilliate/SaveOrder
    [ResponseType(typeof(string))]
    public IHttpActionResult WebHook(ShopifyOrder order)
    {
        // need to return 202 response otherwise webhook is deleted
        return Ok(ProcessOrder(order));
    }
}

Where ProcessOrder loops through the order and saves the details to our internal database.

However if the process takes too long then the webhook calls the api again as it thinks it has failed. Is there any way to return the ok response first but then do the processing after?

Kind of like when you return a redirect in an mvc controller and have the option of continuing with processing the rest of the action after the redirect.

Please note that I will always need to return the ok response as Shopify in all it's wisdom has decided to delete the webhook if it fails 19 times (and processing too long is counted as a failure)

Have you tried to implement a sort of Job, which will process your data on the background and immediately return response to shopify? You can check this answer for details on this: http://stackoverflow.com/questions/14710822/how-to-queue-background-tasks-in-asp-net-web-api — Vsevolod Goloviznin, Nov 21 '14 at 11:42
I'm in the process of turning the method into some sort of asynchronous task to see if that will work, I think the question is more about is there a built in version of returning ok and carrying on rather than having to add a third party library — Pete, Nov 21 '14 at 11:55
Asynchronous processing can help you in improving performance of your operation, but it don't think it will help you in returning the response prior to completion of your operations. Anyway, I'm not aware of any built-in tool for that — Vsevolod Goloviznin, Nov 21 '14 at 12:01

score 16 · Accepted Answer · answered Nov 21 '14 at 13:37

16

I have managed to solve my problem by running the processing asynchronously by using Task:

    // PUT: api/Afilliate/SaveOrder
    public IHttpActionResult WebHook(ShopifyOrder order)
    {
        // this should process the order asynchronously
        var tasks = new[]
        {
            Task.Run(() => ProcessOrder(order))
        };

        // without the await here, this should be hit before the order processing is complete
        return Ok("ok");
    }

answered Nov 21 '14 at 13:37

Pete

57,112
28
117
166

13

Are you OK with `ProcessOrder()` being randomly terminated while it is executing? Because that is the can of worms you open using `Task.Run()` or any other kind of background processing _in your ASP.NET worker process_. If so, then this is the way to go. If not, offload the work to a dedicated program, like a Windows Service. Read the links provided in the [comment by @Vsevolod](http://stackoverflow.com/questions/27060447/web-api-2-return-ok-response-but-continue-processing-in-the-background#comment42635506_27060447). – CodeCaster Nov 21 '14 at 13:39
1

@CodeCaster to be fair it is just a temporary api anyway so I'm not too bothered about writing a whole other service just for processing it and am happy to live with the limitations of using a background process – Pete Nov 21 '14 at 14:25
@CodeCaster can you tell me more about why you said ProcessOrder() might randomnly terminate while executing? – user1431072 Apr 25 '19 at 16:55
@user1431072 because background threads you start yourself are prone to being terminated if your application pool gets recycled. Read the comments and links above. – CodeCaster Apr 25 '19 at 22:23
Just so you know the other comments are correct, the thread will randomly terminate and whatever it was doing, might not complete, if it's long running. I been trying to find a solution to your solution myself. I think CodeCaster, says it best, you have to offload the work to a different program. That's what I'm trying to figure out myself as well – Henry Jul 01 '20 at 23:31

score 6 · Answer 2 · answered Jul 14 '20 at 07:14

There are a few options to accomplish this:

Let a task runner like Hangfire or Quartz run the actual processing, where your web request just kicks off the task.
Use queues, like RabbitMQ, to run the actual process, and the web request just adds a message to the queue... be careful this one is probably the best but can require some significant know-how to setup.
Though maybe not exactly applicable to your specific situation as you are having another process wait for the request to return... but if you did not, you could use Javascript AJAX kick off the process in the background and maybe you can turn retry off on that request... still that keeps the request going in the background so maybe not exactly your cup of tea.

score 2 · Answer 3 · answered Jan 15 '22 at 18:02

I used Response.CompleteAsync(); like below. I also added a neat middleware and attribute to indicate no post-request processing.

[SkipMiddlewareAfterwards]
[HttpPost]
[Route("/test")]
public async Task Test()
{
    /*
       let them know you've 202 (Accepted) the request 
       instead of 200 (Ok), because you don't know that yet.
    */
    HttpContext.Response.StatusCode = 202; 
    await HttpContext.Response.CompleteAsync();
    await SomeExpensiveMethod();
    //Don't return, because default middleware will kick in. (e.g. error page middleware)
}

public class SkipMiddlewareAfterwards : ActionFilterAttribute
{
    //ILB
}

public class SomeMiddleware 
{
    private readonly RequestDelegate next;

    public SomeMiddleware(RequestDelegate next)
    {
        this.next = next;
    }

    public async Task Invoke(HttpContext context)
    {
        await next(context);
        if (context.Features.Get<IEndpointFeature>().Endpoint.Metadata
            .Any(m => m is SkipMiddlewareAfterwards)) return;
        //post-request actions here
    }
}

score 0 · Answer 4 · answered Dec 13 '21 at 18:25

Task.Run(() => ImportantThing() is not an appropriate solution, as it exposes you to a number of potential problems, some of which have already been explained above. Imo, the most nefarious of these are probably unhandled exceptions on the worker process that can actually straight up kill your worker process with no trace of the error outside of event logs or something at captured at the OS, if that's even available. Not good.

There are many more appropriate ways to handle this scenarion, like a handoff a service bus or implementing a HostedService.

https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-6.0&tabs=visual-studio

Web API 2 return OK response but continue processing in the background

4 Answers4

Linked