I've got an Azure app that needs 3 distinctly unique tasks to run as long-running processes. I could easily spin up 3 Worker Roles, but that would cost 3x more than I need to spend given how simple the tasks are. I'd like to run each task in its own thread since each task is very different in the amount of time they take to complete, and how frequently the need to run.
Given that scenario, or even a general multi-threading Azure scenario, what's the best way you know of to have each task run independently?
A few considerations I'd like to propose are:
- Keep the CPU utilization low (not always possible with EventWaitHandles..?).
- Uses the most current framework utilities (i.e. TPL or PLINQ).
- Graceful shutdown and restart in case of an unrecoverable exception.
Thanks for the suggestions, links, or code samples.
EDIT
Here's what I've settled on - Given the code is bound for the Azure cloud, a self-healing option seems the most appropriate. This approach aims to take a collection of threads and delegate their work to a diversity of "units of work."
Here's how it's glued together. First, a simple class to wrap a diversity of workers.
public class ThreadWorker
{
internal void RunInternal()
{
try
{
Run();
}
catch (SystemException)
{
throw;
}
catch (Exception)
{
}
}
public virtual void Run()
{
}
public virtual void OnStop()
{
}
}
Next, the code to keep a series of threads alive indefinitely. This involves a list of threads, a list of "workers" (one per thread), and an infinite loop that checks the status of each thread on a given interval.
Note that an exception that terminates a thread won't tear down the other threads. If an unhandled exception is encountered, the thread will be restarted the next time the event wait handle responds.
private readonly List<Thread> _threads = new List<Thread>();
private readonly List<ThreadWorker> _workers = new List<ThreadWorker>();
private EventWaitHandle EventWaitHandle = new EventWaitHandle(false, EventResetMode.ManualReset);
public override void Run()
{
foreach (var worker in _workers)
{
_threads.Add(new Thread(worker.RunInternal));
}
foreach (var thread in _threads)
{
thread.Start();
}
while (!EventWaitHandle.WaitOne(500))
{
// Restart any threads that have stopped running
for (var i = 0; i < _threads.Count; i++)
{
if (_threads[i].IsAlive)
{
continue;
}
_threads[i] = new Thread(_workers[i].RunInternal);
_threads[i].Start();
}
EventWaitHandle.WaitOne(2000);
}
}
Simple enough - the main thread (in this case an Azure worker role's Run method) blocks for half a second between checks to the active list of threads. Terminated threads are re-started and the host thread then blocks for 2 seconds before checking the worker threads' health again.
Adding workers is as simple as adding a list of classes that inherit from the ThreadWorker base class
_workers.Add(new SomeWorker());
_workers.Add(new AnotherWorker());
And, finally, the concrete workers:
public class SomeWorker: ThreadWorker
{
public override void Run()
{
while (true)
{
// Do your long running work here.
Thread.Sleep(2000); // Take a breather - not necessary.
}
}
}
public class AnotherWorker: ThreadWorker
{
public override void Run()
{
while (true)
{
// Do your long running work here.
}
}
}
