Alternative solution to HostingEnvironment.QueueBackgroundWorkItem in .NET Core

Question

We are working with .NET Core Web Api, and looking for a lightweight solution to log requests with variable intensity into database, but don't want client's to wait for the saving process.
Unfortunately there's no HostingEnvironment.QueueBackgroundWorkItem(..) implemented in dnx, and Task.Run(..) is not safe.
Is there any elegant solution?

Answer 1

As @axelheer mentioned IHostedService is the way to go in .NET Core 2.0 and above.

I needed a lightweight like for like ASP.NET Core replacement for HostingEnvironment.QueueBackgroundWorkItem, so I wrote DalSoft.Hosting.BackgroundQueue which uses.NET Core's 2.0 IHostedService.

PM> Install-Package DalSoft.Hosting.BackgroundQueue

In your ASP.NET Core Startup.cs:

public void ConfigureServices(IServiceCollection services)
{
   services.AddBackgroundQueue(onException:exception =>
   {
                   
   });
}

To queue a background Task just add BackgroundQueue to your controller's constructor and call Enqueue.

public EmailController(BackgroundQueue backgroundQueue)
{
   _backgroundQueue = backgroundQueue;
}
    
[HttpPost, Route("/")]
public IActionResult SendEmail([FromBody]emailRequest)
{
   _backgroundQueue.Enqueue(async cancellationToken =>
   {
      await _smtp.SendMailAsync(emailRequest.From, emailRequest.To, request.Body);
   });

   return Ok();
}

Answer 2

QueueBackgroundWorkItem is gone, but we've got IApplicationLifetime instead of IRegisteredObject, which is being used by the former one. And it looks quite promising for such scenarios, I think.

The idea (and I'm still not quite sure, if it's a pretty bad one; thus, beware!) is to register a singleton, which spawns and observes new tasks. Within that singleton we can furthermore register a "stopped event" in order to proper await still running tasks.

This "concept" could be used for short running stuff like logging, mail sending, and the like. Things, that should not take much time, but would produce unnecessary delays for the current request.

public class BackgroundPool
{
    protected ILogger<BackgroundPool> Logger { get; }

    public BackgroundPool(ILogger<BackgroundPool> logger, IApplicationLifetime lifetime)
    {
        if (logger == null)
            throw new ArgumentNullException(nameof(logger));
        if (lifetime == null)
            throw new ArgumentNullException(nameof(lifetime));

        lifetime.ApplicationStopped.Register(() =>
        {
            lock (currentTasksLock)
            {
                Task.WaitAll(currentTasks.ToArray());
            }

            logger.LogInformation(BackgroundEvents.Close, "Background pool closed.");
        });

        Logger = logger;
    }

    private readonly object currentTasksLock = new object();

    private readonly List<Task> currentTasks = new List<Task>();

    public void SendStuff(Stuff whatever)
    {
        var task = Task.Run(async () =>
        {
            Logger.LogInformation(BackgroundEvents.Send, "Sending stuff...");

            try
            {
                // do THE stuff

                Logger.LogInformation(BackgroundEvents.SendDone, "Send stuff returns.");
            }
            catch (Exception ex)
            {
                Logger.LogError(BackgroundEvents.SendFail, ex, "Send stuff failed.");
            }
        });

        lock (currentTasksLock)
        {
            currentTasks.Add(task);

            currentTasks.RemoveAll(t => t.IsCompleted);
        }
    }
}

Such a BackgroundPool should be registered as a singleton and can be used by any other component via DI. I'm currently using it for sending mails and it works fine (tested mail sending during app shutdown too).

Note: accessing stuff like the current HttpContext within the background task should not work. The old solution uses UnsafeQueueUserWorkItem to prohibit that anyway.

What do you think?

Update:

With ASP.NET Core 2.0 there's new stuff for background tasks, which get's better with ASP.NET Core 2.1: Implementing background tasks in .NET Core 2.x webapps or microservices with IHostedService and the BackgroundService class

Answer 3

You can use Hangfire (http://hangfire.io/) for background jobs in .NET Core.

For example :

var jobId = BackgroundJob.Enqueue(
    () => Console.WriteLine("Fire-and-forget!"));

Answer 4

Here is a tweaked version of Axel's answer that lets you pass in delegates and does more aggressive cleanup of completed tasks.

using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Logging;

namespace Example
{
    public class BackgroundPool
    {
        private readonly ILogger<BackgroundPool> _logger;
        private readonly IApplicationLifetime _lifetime;
        private readonly object _currentTasksLock = new object();
        private readonly List<Task> _currentTasks = new List<Task>();

        public BackgroundPool(ILogger<BackgroundPool> logger, IApplicationLifetime lifetime)
        {
            if (logger == null)
                throw new ArgumentNullException(nameof(logger));
            if (lifetime == null)
                throw new ArgumentNullException(nameof(lifetime));

            _logger = logger;
            _lifetime = lifetime;

            _lifetime.ApplicationStopped.Register(() =>
            {
                lock (_currentTasksLock)
                {
                    Task.WaitAll(_currentTasks.ToArray());
                }

                _logger.LogInformation("Background pool closed.");
            });
        }

        public void QueueBackgroundWork(Action action)
        {
#pragma warning disable 1998
            async Task Wrapper() => action();
#pragma warning restore 1998

            QueueBackgroundWork(Wrapper);
        }

        public void QueueBackgroundWork(Func<Task> func)
        {
            var task = Task.Run(async () =>
            {
                _logger.LogTrace("Queuing background work.");

                try
                {
                    await func();

                    _logger.LogTrace("Background work returns.");
                }
                catch (Exception ex)
                {
                    _logger.LogError(ex.HResult, ex, "Background work failed.");
                }
            }, _lifetime.ApplicationStopped);

            lock (_currentTasksLock)
            {
                _currentTasks.Add(task);
            }

            task.ContinueWith(CleanupOnComplete, _lifetime.ApplicationStopping);
        }

        private void CleanupOnComplete(Task oldTask)
        {
            lock (_currentTasksLock)
            {
                _currentTasks.Remove(oldTask);
            }
        }
    }
}

Answer 5

I know this is a little late, but we just ran into this issue too. So after reading lots of ideas, here's the solution we came up with.

    /// <summary>
    /// Defines a simple interface for scheduling background tasks. Useful for UnitTesting ASP.net code
    /// </summary>
    public interface ITaskScheduler
    {
        /// <summary>
        /// Schedules a task which can run in the background, independent of any request.
        /// </summary>
        /// <param name="workItem">A unit of execution.</param>
        [SecurityPermission(SecurityAction.LinkDemand, Unrestricted = true)]
        void QueueBackgroundWorkItem(Action<CancellationToken> workItem);

        /// <summary>
        /// Schedules a task which can run in the background, independent of any request.
        /// </summary>
        /// <param name="workItem">A unit of execution.</param>
        [SecurityPermission(SecurityAction.LinkDemand, Unrestricted = true)]
        void QueueBackgroundWorkItem(Func<CancellationToken, Task> workItem);
    }


    public class BackgroundTaskScheduler : BackgroundService, ITaskScheduler
    {
        public BackgroundTaskScheduler(ILogger<BackgroundTaskScheduler> logger)
        {
            _logger = logger;
        }

        protected override async Task ExecuteAsync(CancellationToken stoppingToken)
        {
            _logger.LogTrace("BackgroundTaskScheduler Service started.");

            _stoppingToken = stoppingToken;

            _isRunning = true;
            try
            {
                await Task.Delay(-1, stoppingToken);
            }
            catch (TaskCanceledException)
            {
            }
            finally
            {
                _isRunning = false;
                _logger.LogTrace("BackgroundTaskScheduler Service stopped.");
            }
        }

        public void QueueBackgroundWorkItem(Action<CancellationToken> workItem)
        {
            if (workItem == null)
            {
                throw new ArgumentNullException(nameof(workItem));
            }

            if (!_isRunning)
                throw new Exception("BackgroundTaskScheduler is not running.");

            _ = Task.Run(() => workItem(_stoppingToken), _stoppingToken);
        }

        public void QueueBackgroundWorkItem(Func<CancellationToken, Task> workItem)
        {
            if (workItem == null)
            {
                throw new ArgumentNullException(nameof(workItem));
            }

            if (!_isRunning)
                throw new Exception("BackgroundTaskScheduler is not running.");

            _ = Task.Run(async () =>
                {
                    try
                    {
                        await workItem(_stoppingToken);
                    }
                    catch (Exception e)
                    {
                        _logger.LogError(e, "When executing background task.");
                        throw;
                    }
                }, _stoppingToken);
        }

        private readonly ILogger _logger;
        private volatile bool _isRunning;
        private CancellationToken _stoppingToken;
    }

The ITaskScheduler (which we already defined in our old ASP.NET client code for UTest test purposes) allows a client to add a background task. The main purpose of the BackgroundTaskScheduler is to capture the stop cancellation token (which is own by the Host) and to pass it into all the background Tasks; which by definition, runs in the System.Threading.ThreadPool so there is no need to create our own.

To configure Hosted Services properly see this post.

Enjoy!

Answer 6

I have used Quartz.NET (does not require SQL Server) with the following extension method to easily set up and run a job:

public static class QuartzUtils
{
        public static async Task<JobKey> CreateSingleJob<JOB>(this IScheduler scheduler,
            string jobName, object data) where JOB : IJob
        {
            var jm = new JobDataMap { { "data", data } };

            var jobKey = new JobKey(jobName);

            await scheduler.ScheduleJob(
                JobBuilder.Create<JOB>()
                .WithIdentity(jobKey)
                .Build(),

                TriggerBuilder.Create()
                .WithIdentity(jobName)
                .UsingJobData(jm)
                .StartNow()
                .Build());

            return jobKey;
        }
}

Data is passed as an object that must be serializable. Create an IJob that processes the job like this:

public class MyJobAsync :IJob
{
   public async Task Execute(IJobExecutionContext context)
   {
          var data = (MyDataType)context.MergedJobDataMap["data"];
          ....

Execute like this:

await SchedulerInstance.CreateSingleJob<MyJobAsync>("JobTitle 123", myData);

Answer 7

The original HostingEnvironment.QueueBackgroundWorkItem was a one-liner and very convenient to use. The "new" way of doing this in ASP Core 2.x requires reading pages of cryptic documentation and writing considerable amount of code.

To avoid this you can use the following alternative method

    public static ConcurrentBag<Boolean> bs = new ConcurrentBag<Boolean>();

    [HttpPost("/save")]
    public async Task<IActionResult> SaveAsync(dynamic postData)
    {

    var id = (String)postData.id;

    Task.Run(() =>
                {
                    bs.Add(Create(id));
                });

     return new OkResult();

    }


    private Boolean Create(String id)
    {
      /// do work
      return true;
    }

The static ConcurrentBag<Boolean> bs will hold a reference to the object, this will prevent garbage collector from collecting the task after the controller returns.