Simulating message latency accurately

Question

I have written a simple "latency simulator" which works, but at times, messages are delayed for longer than the time specified. I need help to ensure that messages are delayed for the correct amount of time.

The main problem, I believe, is that I am using Thread.Sleep(x), which is depended on various factors but mainly on the clock interrupt rate, which causes Thread.Sleep() to have a resolution of roughly 15ms. Further, intensive tasks will demand more CPU time and will occasionally result in a delay greater than the one requested. If you are not familiar with the resolution issues of Thread.Sleep, you can read these SO posts: here, here and here.

This is my LatencySimulator:

public class LatencySimulatorResult: EventArgs
{
    public int messageNumber { get; set; }
    public byte[] message { get; set; }
}

public class LatencySimulator
{
    private int messageNumber;
    private int latency = 0;
    private int processedMessageCount = 0;

    public event EventHandler messageReady;

    public void Delay(byte[] message, int delay)
    {
        latency = delay;

        var result = new LatencySimulatorResult();
        result.message = message;
        result.messageNumber = messageNumber;

        if (latency == 0)
        {
            if (messageReady != null)
                messageReady(this, result);
        }
        else
        {
            ThreadPool.QueueUserWorkItem(ThreadPoolCallback, result);
        }
        Interlocked.Increment(ref messageNumber);
    }

    private void ThreadPoolCallback(object threadContext)
    {
        Thread.Sleep(latency);
        var next = (LatencySimulatorResult)threadContext;

        var ready = next.messageNumber == processedMessageCount + 1;
        while (ready == false)
        {
            ready = next.messageNumber == processedMessageCount + 1;
        }

        if (messageReady != null)
            messageReady(this, next);

        Interlocked.Increment(ref processedMessageCount);
    }
}

To use it, you create a new instance and bind to the event handler:

var latencySimulator = new LatencySimulator();
latencySimulator.messageReady += MessageReady;

You then call latencySimulator.Delay(someBytes, someDelay); When a message has finished being delayed, the event is fired and you can then process the delayed message.

It is important that the order in which messages are added is maintained. I cannot have them coming out the other end of the latency simulator in some random order.

Here is a test program to use the latency simulator and to see how long messages have been delayed for:

private static LatencySimulator latencySimulator;
private static ConcurrentDictionary<int, PendingMessage> pendingMessages;
private static List<long> measurements;

static void Main(string[] args)
{
    var results = TestLatencySimulator();
    var anomalies = results.Result.Where(x=>x > 32).ToList();
    foreach (var result in anomalies)
    {
        Console.WriteLine(result);
    }

    Console.ReadLine();
}

static async Task<List<long>> TestLatencySimulator()
{
    latencySimulator = new LatencySimulator();
    latencySimulator.messageReady += MessageReady;
    var numberOfMeasurementsMax = 1000;
    pendingMessages = new ConcurrentDictionary<int, PendingMessage>();
    measurements = new List<long>();

    var sendTask = Task.Factory.StartNew(() =>
    {
        for (var i = 0; i < numberOfMeasurementsMax; i++)
        {
            var message = new Message { Id = i };
            pendingMessages.TryAdd(i, new PendingMessage() { Id = i });
            latencySimulator.Delay(Serialize(message), 30);
            Thread.Sleep(50);
        }
    });

    //Spin some tasks up to simulate high CPU usage
    Task.Factory.StartNew(() => { FindPrimeNumber(100000); });
    Task.Factory.StartNew(() => { FindPrimeNumber(100000); });
    Task.Factory.StartNew(() => { FindPrimeNumber(100000); });

    sendTask.Wait();

    return measurements;
}

static long FindPrimeNumber(int n)
{
    int count = 0;
    long a = 2;
    while (count < n)
    {
        long b = 2;
        int prime = 1;// to check if found a prime
        while (b * b <= a)
        {
            if (a % b == 0)
            {
                prime = 0;
                break;
            }
            b++;
        }
        if (prime > 0)
        {
            count++;
        }
        a++;
    }
    return (--a);
}

private static void MessageReady(object sender, EventArgs e)
{
    LatencySimulatorResult result = (LatencySimulatorResult)e;

    var message = (Message)Deserialize(result.message);
    if (pendingMessages.ContainsKey(message.Id) != true) return;

    pendingMessages[message.Id].stopwatch.Stop();
    measurements.Add(pendingMessages[message.Id].stopwatch.ElapsedMilliseconds);
}

static object Deserialize(byte[] arrBytes)
{
    using (var memStream = new MemoryStream())
    {
        var binForm = new BinaryFormatter();
        memStream.Write(arrBytes, 0, arrBytes.Length);
        memStream.Seek(0, SeekOrigin.Begin);
        var obj = binForm.Deserialize(memStream);
        return obj;
    }
}

static byte[] Serialize<T>(T obj)
{
    BinaryFormatter bf = new BinaryFormatter();
    using (var ms = new MemoryStream())
    {
        bf.Serialize(ms, obj);
        return ms.ToArray();
    }
}

If you run this code, you will see that about 5% of the messages are delayed for more than the expected 30ms. In fact, some are as high as 60ms. Without any background tasks or high CPU usage, the simulator behaves as expected.

I need them all to be 30ms (or as close to as possible) - I do not want some arbitrary 50-60ms delays.

Can anyone suggest how I can refactor this code so that I can achieve the desired result, but without the use of Thread.Sleep() and with as little CPU overhead as possible?