How to handle multiple tasks running in parallel at different intervals inside a C# based Windows service?

Question

I already have some experience in working with threads in Windows but most of that experience comes from using Win32 API functions in C/C++ applications. When it comes to .NET applications however, I am often not sure about how to properly deal with multithreading. There are threads, tasks, the TPL and all sorts of other things I can use for multithreading but I never know when to use which of those options. I am currently working on a C# based Windows service which needs to periodically validate different groups of data from different data sources. Implementing the validation itself is not really an issue for me but I am unsure about how to handle all of the validations running simultaneously. I need a solution for this which allows me to do all of the following things:

1. Run the validations at different (predefined) intervals.
2. Control all of the different validations from one place so I can pause and/or stop them if necessary, for example when a user stops or restarts the service.
3. Use the system ressources as efficiently as possible to avoid performance issues.

So far I've only had one similar project before where I simply used Thread objects combined with a ManualResetEvent and a Thread.Join call with a timeout to notify the threads about when the service is stopped. The logic inside those threads to do something periodically then looked like this:

while (!shutdownEvent.WaitOne(0))
{
    if (DateTime.Now > nextExecutionTime)
    {
        // Do something
        nextExecutionTime = nextExecutionTime.AddMinutes(interval);
    }

    Thread.Sleep(1000);
}

While this did work as expected, I've often heard that using threads directly like this is considered "oldschool" or even a bad practice. I also think that this solution does not use threads very efficiently as they are just sleeping most of the time. How can I achive something like this in a more modern and efficient way? If this question is too vague or opinion-based then please let me know and I will try my best to make it as specific as possible.

Answer 1

Question feels a bit broad but we can use the provided code and try to improve it.

Indeed the problem with the existing code is that for the majority of the time it holds thread blocked while doing nothing useful (sleeping). Also thread wakes up every second only to check the interval and in most cases go to sleep again since it's not validation time yet. Why it does that? Because if you will sleep for longer period - you might block for a long time when you signal shutdownEvent and then join a thread. Thread.Sleep doesn't provide a way to be interrupted on request.

To solve both problems we can use:

1. Cooperative cancellation mechanism in form of CancellationTokenSource + CancellationToken.

2. Task.Delay instead of Thread.Sleep.

For example:

async Task ValidationLoop(CancellationToken ct) {
    while (!ct.IsCancellationRequested) {
        try {
            var now = DateTime.Now;
            if (now >= _nextExecutionTime) {
                // do something
                _nextExecutionTime = _nextExecutionTime.AddMinutes(1);
            }

            var waitFor = _nextExecutionTime - now;
            if (waitFor.Ticks > 0) {
                await Task.Delay(waitFor, ct);
            }
        }
        catch (OperationCanceledException) {
            // expected, just exit
            // otherwise, let it go and handle cancelled task 
            // at the caller of this method (returned task will be cancelled).
            return;
        }
        catch (Exception) {
            // either have global exception handler here
            // or expect the task returned by this method to fail
            // and handle this condition at the caller
        }
    }
}

Now we do not hold a thread any more, because await Task.Delay doesn't do this. Instead, after specificed time interval it will execute the subsequent code on a free thread pool thread (it's more complicated that this but we won't go into details here).

We also don't need to wake up every second for no reason, because Task.Delay accepts cancellation token as a parameter. When that token is signalled - Task.Delay will be immediately interrupted with exception, which we expect and break from the validation loop.

To stop the provided loop you need to use CancellationTokenSource:

private readonly CancellationTokenSource _cts = new CancellationTokenSource();

And you pass its _cts.Token token into the provided method. Then when you want to signal the token, just do:

_cts.Cancel();

To futher improve the resource management - IF your validation code uses any IO operations (reads files from disk, network, database access etc) - use Async versions of said operations. Then also while performing IO you will hold no unnecessary threads blocked waiting.

Now you don't need to manage threads yourself anymore and instead you operatate in terms of tasks you need to perform, letting framework \ OS manage threads for you.

Answer 2

You should use Microsoft's Reactive Framework (aka Rx) - NuGet System.Reactive and add using System.Reactive.Linq; - then you can do this:

Subject<bool> starter = new Subject<bool>();

IObservable<Unit> query =
    starter
        .StartWith(true)
        .Select(x => x
            ? Observable.Interval(TimeSpan.FromSeconds(5.0)).SelectMany(y => Observable.Start(() => Validation()))
            : Observable.Never<Unit>())
        .Switch();
        
IDisposable subscription = query.Subscribe();

That fires off the Validation() method every 5.0 seconds.

When you need to pause and resume, do this:

starter.OnNext(false);
// Now paused

starter.OnNext(true);
// Now restarted.

When you want to stop it all call subscription.Dispose().