How to lock equal requests to an external service in C#?

Question

Let's say I have a scoped service called IPackageLoader with one simple method GetAsync(string packageName) that enables us to retrieve a package based on one HTTP GET call.

Since it is a HTTP GET call, it is idempotent, so there's no point in making multiple requests if GetAsync(string packageName) gets called multiple times.

Example:

packageLoader.GetAsync("dotnet/aspnetcore")

packageLoader.GetAsync("dotnet/aspnetcore")

packageLoader.GetAsync("dotnet/efcore")

If there isn't some lock mechanism here, the second request will be made while it could just wait for the first one, save it and return it. However, the third request, since it is to get some other package, could be made without awaiting.

I tried to use SemaphoreSlim but I failed, since I locked on the first request, and the second one would wait for the first one to arrive as desired. However, if I tried to get a different package, using SemaphoreSlim I'll need to wait unnecessarily.

Any ideas on how to achieve this behavior? Maybe some class from the library that allows me to lock based on some key - instead just locking while any request is being done?

Answer 1

You should cache the request(packageName) also the actual task (not the result)

simplified e.g:

var cache = new CustomCache();
var loader = new PackageLoader(cache);

// simulating 
var t1 = loader.GetAsync("lib1");
var t2 = loader.GetAsync("lib1");
var t3 = loader.GetAsync("lib2");
var t4 = loader.GetAsync("lib1");

await Task.WhenAll(t1,t2,t3,t4);

// lifetime scope
class PackageLoader
{
    private CustomCache _cache;
    
    public PackageLoader(CustomCache cache)
    {
        _cache = cache;
    }
    public Task GetAsync(string packageName)
    {
        return _cache.Dic.GetOrAdd(packageName, q => HttpCall(q));
    }
    
    private async Task HttpCall(string packageName)
    {
        var rand = new Random();    
        // simulating a http call
        await Task.Delay(rand.Next(1, 3) * 1000);
        Console.WriteLine($"{packageName} is loaded");
    }
}

// lifetime singleton
class CustomCache
{
    public ConcurrentDictionary<string,Task> Dic = new();  
    
    // some cleanup mechanism, probably in a time preiod
}

this way you only have 1 HTTP call for each package. just make sure to clean the cache when the task completed

lib1 is loaded
lib2 is loaded

Answer 2

I think I was able to do it using SemaphoreSlim and a Dictionary (this is an example using a Console Application .NET 6 template - no Main method needed):

var dataLoader = new DataLoader();

Console.WriteLine("Executing...");

var t1 = dataLoader.LoadAsync("1");
var t2 = dataLoader.LoadAsync("1");
var t3 = dataLoader.LoadAsync("9999");

Console.WriteLine("Done Loading...");

Console.WriteLine(t1 == t2);
Console.WriteLine(t1 == t3);

Console.WriteLine(await t1);
Console.WriteLine(await t2);
Console.WriteLine(await t3);

public class DataLoader
{
    private readonly SemaphoreSlim semaphoreSlim = new SemaphoreSlim(1, 1);

    private Dictionary<string, Task<string>> Data { get; set; }

    public DataLoader()
    {
        this.Data = new Dictionary<string, Task<string>>();
    }

    public Task<string> LoadAsync(string id)
    {
        this.semaphoreSlim.Wait();

        try
        {
            if (this.Data.TryGetValue(id, out var dataTask))
            {
                return dataTask;
            }

            dataTask = GetString(id);

            this.Data.Add(id, dataTask);

            return dataTask;
        }
        finally
        {
            this.semaphoreSlim.Release();
        }
    }

    private async Task<string> GetString(string str)
    {
        await Task.Delay(5000);

        return str;
    }
}

Here's the result (notice that t1 is equal to t2):

Executing...
Done Loading...
True
False
1
1
9999