2

I am trying to read and process some data from Redis in parallel. However, after printing various time stamps of when operations start and end and measuring such times it seems as if the tasks do not happen in parallel.

I've extracted a small console app (from which I removed the console prints):

public async Task WorkAsync()
{
    var redis = ConnectionMultiplexer.Connect(...);
    var database = redis.GetDatabase();

    Task<HashEntry[]> task1 = Task.Run(async () =>
        {
            return await database.HashGetAllAsync("some stored hash key");
        });

    Task<HashEntry[]> task2 = Task.Run(async () =>
        {
            return await database.HashGetAllAsync("another stored hash key");
        });

    await Task.WhenAll(new[] { task1, task2 });
}

What I notice here is that the two tasks arrive at the HashGetAllAsync method calls virtually simultaneously (so parallelism OK so far) but then somehow there seems to be some contention over the database or multiplexer object. When commenting each of the tasks in turn, they each take roughly the same time to complete. When I run both of them as in the example above, the second one that finishes will always take much more time than the first (giving me a strong feeling that the second Redis call somehow waits for the first to complete).

If I change the code so I make the multiplexer a local variable inside each task then they will behave as I expect (they finish about the same moment).

public async Task WorkAsync()
{
    Task<HashEntry[]> task1 = Task.Run(async () =>
        {
            var redis = ConnectionMultiplexer.Connect(...);
            var database = redis.GetDatabase();

            return await database.HashGetAllAsync("some stored hash key");
        });

    Task<HashEntry[]> task2 = Task.Run(async () =>
        {
            var redis = ConnectionMultiplexer.Connect(...);
            var database = redis.GetDatabase();

            return await database.HashGetAllAsync("another stored hash key");
        });

    await Task.WhenAll(new[] { task1, task2 });
}

I know the documentation says that one should really not make more multiplexer instances but just one and store it, and that the multiplexer is optimized for heavily concurrent access.

I don't know how to validate that the calls to the Redis server actually "leave" the Redis library at the same time (I tried procmon but instead of just 2 TCP events I see much more so that didn't clear up much).

I really can't figure out why this behavior or whether I am not going about the best way of getting and processing Redis data in parallel.

CyberDude
  • 8,541
  • 5
  • 29
  • 47
  • 1
    You know that redis has a single threaded access model, right? – Yuval Itzchakov Feb 15 '15 at 14:07
  • Actually I seem to have overlooked this detail... now that you mention it – CyberDude Feb 15 '15 at 14:13
  • [This](http://stackoverflow.com/questions/10489298/redis-is-single-threaded-then-how-does-it-do-concurrent-i-o) will also probably be of use. – Yuval Itzchakov Feb 15 '15 at 14:14
  • Yes, I am starting to digest this aspect but I'm also thinking what to do with get operations that take longer (because of large cached values). – CyberDude Feb 15 '15 at 14:24
  • And you're thinking that paralleling the work will make it faster? – Yuval Itzchakov Feb 15 '15 at 14:26
  • Well if each read takes 1000ms and the local processing of that data takes 100ms each then I hope to be able to finish the entire work in roughly 1100ms instead of 2200ms – CyberDude Feb 15 '15 at 14:33
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/70958/discussion-between-cyberdude-and-yuval-itzchakov). – CyberDude Feb 15 '15 at 14:36
  • Actually, I wonder whether Task.Run is part of the delay here (with spin-up etc); there isn't any inherent synchronisation here - the library is designed to behave as you expect - essentially writing both requests to the stream without waiting for any replies. To confirm: it is not "request 1 response 1 request 2 response 2..."; responses are *completely* separate to requests (except for preserving order) – Marc Gravell Feb 15 '15 at 20:15

0 Answers0