Performing a health check in .NET Core Worker Service

Question

How can I implement health checks in a .NET Core Worker Service?

The service will be run inside Docker and needs to be able to check the health of the service.

Answer 1

Another way of doing this is to implement IHealthCheckPublisher.

The benefits of this approach is the ability to re-use your existing IHealthChecks or integration with 3rd party libraries that rely on IHealthCheck interface (like this one).

Though you still target Microsoft.NET.Sdk.Web as the SDK you don't need to add any asp.net specifics.

Here is an example:

public static IHostBuilder CreateHostBuilder(string[] args)
{
  return Host
    .CreateDefaultBuilder(args)
    .ConfigureServices((hostContext, services) =>
    {
      services
        .AddHealthChecks()
        .AddCheck<RedisHealthCheck>("redis_health_check")
        .AddCheck<RfaHealthCheck>("rfa_health_check");

      services.AddSingleton<IHealthCheckPublisher, HealthCheckPublisher>();
      services.Configure<HealthCheckPublisherOptions>(options =>
      {
        options.Delay = TimeSpan.FromSeconds(5);
        options.Period = TimeSpan.FromSeconds(5);
      });
    });
}

public class HealthCheckPublisher : IHealthCheckPublisher
{
  private readonly string _fileName;
  private HealthStatus _prevStatus = HealthStatus.Unhealthy;

  public HealthCheckPublisher()
  {
    _fileName = Environment.GetEnvironmentVariable(EnvVariableNames.DOCKER_HEALTHCHECK_FILEPATH) ??
                Path.GetTempFileName();
  }

  public Task PublishAsync(HealthReport report, CancellationToken cancellationToken)
  {
    // AWS will check if the file exists inside of the container with the command
    // test -f $DOCKER_HEALTH_CHECK_FILEPATH

    var fileExists = _prevStatus == HealthStatus.Healthy;

    if (report.Status == HealthStatus.Healthy)
    {
      if (!fileExists)
      {
        using var _ = File.Create(_fileName);
      }
    }
    else if (fileExists)
    {
      File.Delete(_fileName);
    }

    _prevStatus = report.Status;

    return Task.CompletedTask;
  }
}

Answer 2

I don't think is worth it to change SDK to Microsoft.NET.Sdk.Web. You will include additional middlewares just because of one health check? No thanks ...

What you could do is to use a different protocol like TCP.

The general idea is:

1. Create a separate background service that creates a TCP server (take a look at TcpListener.cs)
2. When you receive a request you have two options: if the application is healthy accept TCP connection otherwise reject it.
3. If you use containers your orchestrator should have an option to call it over TCP (in k8s there is a property tcpSocket)

If you need more detailed information you may check: Monitoring Health of ASP.NET Core Background Services With TCP Probes on Kubernetes

Cheers!

Answer 3

I think that you should also consider to retain the Microsoft.NET.Sdk.Worker.

Don't change the whole sdk just because of the health checks.

Then you can create a backgroundservice (just like the main worker), in order to update a file to write for example the current timestamp. An example of the background health check worker would be:

public class HealthCheckWorker : BackgroundService
{
    private readonly int _intervalSec;
    private readonly string _healthCheckFileName;

    public HealthCheckWorker(string healthCheckFileName, int intervalSec)
    {
        this._intervalSec = intervalSec;
        this._healthCheckFileName = healthCheckFileName;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (true)
        {
            File.WriteAllText(this._healthCheckFileName, DateTime.UtcNow.ToString());
            await Task.Delay(this._intervalSec * 1000, stoppingToken);
        }
    }
}

Then you can add a extension method like this:

public static class HealthCheckWorkerExtensions
{
    public static void AddHealthCheck(this IServiceCollection services,
        string healthCheckFileName, int intervalSec)
    {
        services.AddHostedService<HealthCheckWorker>(x => new HealthCheckWorker(healthCheckFileName, intervalSec));
    }
}

With this you can add in services the health check support

.ConfigureServices(services =>
{
    services.AddHealthCheck("hc.txt", 5);
})

Answer 4

Add HTTPListener and expose the health checks endpoints.

Using HTTPListener does not require adding Microsoft.NET.Sdk.Web SDK.

Program.cs

    using Consumer;
    
    IHost host = Host.CreateDefaultBuilder(args)
        .ConfigureServices(services =>
        {
            services.AddHostedService<Worker>();
            services.AddHostedService<HttpHealthcheck>();
        })
        .Build();
    
    await host.RunAsync();

HttpHealthcheck.cs

    using System.Net;
    using System.Text;
    
    namespace Consumer;
    
    public class HttpHealthcheck : BackgroundService
    {
        private readonly ILogger<Worker> _logger;
        private readonly HttpListener _httpListener;
        private readonly IConfiguration _configuration;
    
    
        public HealthcheckHttpListener(ILogger<Worker> logger, IConfiguration configuration)
        {
            _logger = logger;
            _configuration = configuration;
            _httpListener = new HttpListener();
        }
    
    
        protected override async Task ExecuteAsync(CancellationToken stoppingToken)
        {
    
            _httpListener.Prefixes.Add($"http://*:5001/healthz/live/");    
            _httpListener.Prefixes.Add($"http://*:5001/healthz/ready/");
    
            _httpListener.Start();
            _logger.LogInformation($"Healthcheck listening...");
    
            while (!stoppingToken.IsCancellationRequested)
            {
                HttpListenerContext ctx = null;
                try
                {
                    ctx = await _httpListener.GetContextAsync();
                }
                catch (HttpListenerException ex)
                {
                    if (ex.ErrorCode == 995) return;
                }
    
                if (ctx == null) continue;
    
                var response = ctx.Response;
                response.ContentType = "text/plain";
                response.Headers.Add(HttpResponseHeader.CacheControl, "no-store, no-cache");
                response.StatusCode = (int)HttpStatusCode.OK;
    
                var messageBytes = Encoding.UTF8.GetBytes("Healthy");
                response.ContentLength64 = messageBytes.Length;
                await response.OutputStream.WriteAsync(messageBytes, 0, messageBytes.Length);
                response.OutputStream.Close();
                response.Close();
            }
        }
    }

Answer 5

What I've done to accomplish this is add Microsoft.NET.Sdk.Web to my Worker, and then configured a web host to run alongside the worker:

Host.CreateDefaultBuilder(args)
    .ConfigureWebHostDefaults(builder =>
    {
        builder.UseStartup<Startup>();
    })
    .ConfigureServices((hostContext, services) =>
    {
        services.AddHostedService<Worker>();
        services.AddLogging(builder =>
            builder
                .AddDebug()
                .AddConsole()
        );
    });

With that done, all that's left to do is map the health check endpoint as you normally would with ASP.NET Core.