20

Trying to use the await keyword in a LINQ query and I get this:

The 'await' operator may only be used in a query expression within the first collection expression of the initial 'from' clause or within the collection expression of a 'join' clause

Sample Code:

var data = (from id in ids
            let d = await LoadDataAsync(id)
            select d);

Is it not possible to await something in a LINQ query, or does it need to be structured a different way?

Luke Puplett
  • 42,091
  • 47
  • 181
  • 266
c0D3l0g1c
  • 3,020
  • 5
  • 33
  • 71
  • I guess there is too much compiler magic involved here, you need to structure this differently, just write it out as a normal foreach-loop. – Lasse V. Karlsen May 18 '13 at 13:15

3 Answers3

26

LINQ has very limited support for async/await. For LINQ-to-objects, the only really useful operation I know of is to do a Select with an async delegate (which results in a sequence of tasks).

List<T> data = new List<T>();
foreach (var id in ids)
  data.Add(await LoadDataAsync(id));

If you can do LoadDataAsync in parallel safely, your example could be rewritten as:

T[] data = await Task.WhenAll(ids.Select(id => LoadDataAsync(id)));
Luke Puplett
  • 42,091
  • 47
  • 181
  • 266
Stephen Cleary
  • 437,863
  • 77
  • 675
  • 810
  • Would this perform better than a Parallel.ForEach loop? – c0D3l0g1c May 18 '13 at 13:45
  • 4
    It would if `LoadData` is I/O-bound. – Stephen Cleary May 18 '13 at 13:56
  • Wonderfull! LoadData is an I/O procedure. – c0D3l0g1c May 18 '13 at 14:02
  • I edited to rename `LoadData` to `LoadDataAsync` it then makes it clearer that without the `await` this method returns a `Task` and as such the `WhenAll` method will work. – Luke Puplett May 06 '14 at 12:33
  • @StephenCleary What's so special with `Select` that _"only really useful operation I know of is to do a Select with an async delegate"_ as opposed to `Where` ? – Royi Namir Jan 29 '15 at 13:15
  • @RoyiNamir: `Select` is a projection, and I'm using it to project to a sequence of *tasks*. Then the calling code (outside LINQ) can use `WhenAll` to properly (asynchronously) wait for those tasks to complete. `Where` is a filter; more to the point, it's a *synchronous* filter. So there's no way to do an async `Where` without doing sync-over-async, with all the problems that entails. Note that IObservable is by nature asynchronous, and you can write a `WhereAsync` for Rx. – Stephen Cleary Jan 29 '15 at 13:33
  • Also note that this `ids.Select(id => LoadDataAsync(id))` could be written as `ids.Select(LoadDataAsync)`. Which notation you use is of no importance and just a matter of personal taste. – T_D May 14 '20 at 15:56
3

You can define some async linq operations by yourself (for linq to objects): for example: you can write your own WhereAsync extension method:

public static async Task<IEnumerable<T>> WhereAsync<T>(
this IEnumerable<T> target, Func<T, Task<bool>> predicateAsync)
{
   var tasks = target.Select(async x => new { Predicate = await predicateAsync(x).ConfigureAwait(false), Value = x }).ToArray();
   var results = await Task.WhenAll(tasks).ConfigureAwait(false);

   return results.Where(x => x.Predicate).Select(x => x.Value);
}

And use it like that:

var ints = new List<int> { 1, 2, 3 };
var smallInts = await ints.WhereAsync(IsSmallIntAsync);
GFoley83
  • 3,439
  • 2
  • 33
  • 46
quadroid
  • 8,444
  • 6
  • 49
  • 82
  • 1
    This approach uses sync-over-async in a way that is [prone to deadlock](http://blog.stephencleary.com/2012/07/dont-block-on-async-code.html) (as I explain on my blog). – Stephen Cleary Jan 29 '15 at 13:01
  • @StephenCleary yeah i see, i hate the ConfigureAwait code litering, don't know why the default does use the context at all. I fixed that. – quadroid Jan 29 '15 at 13:07
  • It's still prone to deadlock unless the user always uses `ConfigureAwait` within their predicate. And only uses libraries that use `ConfigureAwait` (i.e., HttpClient doesn't on some platforms). – Stephen Cleary Jan 29 '15 at 13:12
  • @StephenCleary Ì do not understand whats the problem, If the user blocks the context he will get a deadlock (this is nothing new, neither a problem of async/await, and will deadlock a application using the SynchronizationContext aswell). And i do not know how i could change this anyway. As you already posted HttpClient will also deadlock if the user blocks the context. The root problem is the context block, not my method. Am i wrong? – quadroid Jan 29 '15 at 13:20
  • The root problem is that your method is calling `Task.Result` instead of using `await`, because it's trying to do sync-over-async. This approach will work in some limited scenarios, but this `WhereAsync` method is **not** a general-purpose solution because it can cause that deadlock. – Stephen Cleary Jan 29 '15 at 13:25
  • 1
    @StephenCleary i did await the Task before i accessed the result, i thought this would change the behavior of task.Result to be non blocking. How can i await on all tasks and access the results afterwards? – quadroid Jan 29 '15 at 13:29
  • 2
    Sorry, I totally missed that first `await`. I still recommend that you use `await` to retrieve the results to avoid `AggregateException`, but the code does not have the sync-over-async problem. – Stephen Cleary Jan 29 '15 at 13:35
  • Could you please tell me how can I use the `WhereAsync` method you wrote here, to implement an `Except` method which accepts `Func` as one of the input parameters? – Mo Sadeghipour Jan 17 '18 at 09:07
0

Using reactive extensions, it is possible to handle the results of a linq query asynchronously like this:

(from d in ids
select LoadDataAsync(d).ToObservable()).Merge()

This is gives you an observable stream you can respond to in various ways. For example, you can then .Buffer the results into a list with a timeout.

The above in essence says "for every d in ids, apply an asynchronous function to it, which yields a task for each d, and treat that as an observable of an individual result (ToObservable), and treat all those observables together as a single observable stream (Merge)

Sentinel
  • 3,582
  • 1
  • 30
  • 44