Using Interlocked.Add to Task.Add when using async await

Question

I am trying to convert the following code to async/await:

long deletedBlobCount = 0;
Parallel.ForEach(BlobURLList, item => Interlocked.Add(ref deletedBlobCount, DeleteBlob(item)));

DeleteBlob(item) function returns 1 if blob was deleted, 0 otherwise. This is what my new code looks like (I am on .Net 4):

long deletedBlobCount = 0;
var deleteBlobTask = new List<Task>();
foreach (var item in BlobURLList)
{
    deleteBlobTask.Add(t => { Interlocked.Add(ref deletedBlobCount, await DeleteBlobAsync(item))});
}

Compiler doesn't like await DeleteBlobAsync(item) here. This is the error: The 'await' operator can only be used within an async lambda expression. Consider marking this lambda expression with the 'async' modifier

What am I missing? Also, I can't make deletedBlobCount a class level variable (requirement for logging).

What do you mean by "doesn't like"? What's the compiler error message? — Gabriel Luci, Feb 03 '22 at 01:29
Does this answer your question? [The 'await' operator can only be used within an async lambda expression](https://stackoverflow.com/questions/20593501/the-await-operator-can-only-be-used-within-an-async-lambda-expression). The issue is unrelated to Interlocked.Add. The issue is the presence of `await` in a non-async lambda. So make the lambda async. — Raymond Chen, Feb 03 '22 at 01:52
The second piece of code seems like it will block on `Interlocked.Add` before it processes the next item from `BloblURLList`. — ProgrammingLlama, Feb 03 '22 at 02:01

Gabriel Luci · Accepted Answer · 2022-02-03T02:37:09.503

2

Your code has several problems. That specific compiler error is complaining that you're using await inside a lambda function that is not marked async. It's expecting something like this:

deleteBlobTask.Add(async () => { Interlocked.Add(ref deletedBlobCount, await DeleteBlobAsync(item)); });

I also got rid of the t parameter, since you're not using it.

Notice that you were also missing a semicolon before the closing }. In a one-liner lambda, you can only omit the semicolon if you omit the braces, like this:

deleteBlobTask.Add(async () => Interlocked.Add(ref deletedBlobCount, await DeleteBlobAsync(item)) );

However, that still won't compile because the lambda expression is a Func<Task>: a function that returns a Task. Really what you want is a Task, or in other words, you want the result of running that function, not the function itself. As you have it, that function isn't being run.

To declare an inline function (a lambda) and run it at the same time, the syntax gets a little convoluted:

deleteBlobTask.Add(new Func<Task>(async () => Interlocked.Add(ref deletedBlobCount, await DeleteBlobAsync(item)))() );

Notice that you declare the Func<Task>, then put () at the end, just like you would any method call.

So.... it's complicated. Are you sure all this is that much better than just:

foreach (var item in BlobURLList)
{
    deletedBlobCount += await DeleteBlobAsync(item);
}

edited Feb 03 '22 at 02:37

answered Feb 03 '22 at 02:10

Gabriel Luci

38,328
4
55
84

Isn't using `Interlocked` in the last example pointless? – ProgrammingLlama Feb 03 '22 at 02:12
1

One benefit of the original version is that the tasks run in parallel. – Raymond Chen Feb 03 '22 at 02:13
@RaymondChen True, but does any actual performance benefit outweigh the lack of readable code? That's a question only the OP can answer. Either way gets the job done. – Gabriel Luci Feb 03 '22 at 02:14
Agree with what @RaymondChen said. I plan to call Task.WhenAll(deleteTask) after that line. My blobs are big, so deletion takes time. Parallelism is what I need. – Gaurav Feb 03 '22 at 02:15
3

You can write a helper lambda: `Func maker = async item => Interlocked.Add(ref deletedBlobCount, await DeleteBlobAsync(item));`. Then it becomes `deleteBlobTask.Add(maker(item));` inside the loop. – Raymond Chen Feb 03 '22 at 02:17
@Gaurav Keep in mind that this code is asynchronous, but not parallel. Parallel is running two parts of code at the same time using multiple threads. Asynchronous is freeing a thread to do other work while it waits for something external. When `DeleteBlobAsync` actually makes the I/O request, your code will move on to the next one. When the response returns, each task will complete, but all on the same thread. That can speed things up, sometimes. Sometimes it actually slows you down, since there is overhead to asynchronous code. Sometimes it's worth testing if it actually helps. – Gabriel Luci Feb 03 '22 at 02:21
@RaymondChen That would definitely be more readable. – Gabriel Luci Feb 03 '22 at 02:25
@Llama Maybe. It depends on what type of project this is, and if there is a synchronization context. If there is (like in ASP.NET - not Core - or desktop app) then all of this is run on the same thread and there is no value to using `Interlocked`. If there is no synchronization context (ASP.NET Core or a console app), then the continuations of the tasks (everything after the `await`, which includes the call to `Interlocked.Add`) can run on different threads in parallel. – Gabriel Luci Feb 03 '22 at 02:30
1

Note: we're still talking about the last example. `deletedBlobCount` is a local variable, and the only async method is `await DeleteBlobAsync(1);` which will be evaluated _before_ being passed to `Interlocked.Add`, so the code will actually block in that section anyway, meaning that it's only being incremented from one place at a time. I'm not saying that that code won't be resumed on a different thread, but the second iteration of the loop won't run until after `Interlocked.Add` completes, and so on. `deletedBlobCount += await DeleteBlobAsync(1)` is fine for the last example. – ProgrammingLlama Feb 03 '22 at 02:32
1

@Llama Oh, the last example, yeah. You're right. There's no value to using `Interlocked` there. – Gabriel Luci Feb 03 '22 at 02:36
Thank you @GabrielLuci, I am using lambda that RaymondChen wrote along with custom implementation of ForEachAsync (I am using .Net 4). I picked custom implementation from here: https://github.com/dotnet/runtime/issues/1946 Can you please explain more on why next iteration of Task.WhenAll stops until Interlocked.Add is completed? – Gaurav Feb 03 '22 at 15:25
@Gaurav I can't say for sure without seeing your new code. But if `Interlocked.Add` is inside the `ForEachAsync`, then the task won't complete until `Interlocked.Add` completes, which is normal. – Gabriel Luci Feb 03 '22 at 17:27

score 0 · Answer 2 · edited Feb 03 '22 at 02:09

0

Try this in an async method:

long deletedBlobCount = 0;
var deleteBlobTask = new List<Task>();

foreach (var item in BlobURLList)
{
    deleteBlobTask.Add(((Func<Task>)(async () => { 
        long sum = await DeleteBlobAsync(item);
        Interlocked.Add(ref deletedBlobCount, sum);
    }))());
}

edited Feb 03 '22 at 02:09

Raymond Chen

44,448
11
96
135

answered Feb 03 '22 at 01:57

Rivo R.

1
2

Using Interlocked.Add to Task.Add when using async await

2 Answers2