Why does this ParallelForEachAsync method never return?

Question

I try to execute this code async and parallel by using the ParallelForEachAsync method from this project: Dasync/AsyncEnumerable. Unfortunately the method never returns. The SampleProduct is a simple DTO which has a boolean property and two string properties. The GetOsmData method tries to get data via http-request, but often throws an exception.

My first attempt was without .ConfigureAwait(false), but has the same result...

If I try this method with a list of products (products.Count = 8) result.Count always stops by 7.

private async Task<ConcurrentBag<SampleProduct>> CheckOsmDataAsync(
    List<SampleProduct> products)
{
    var result = new ConcurrentBag<SampleProduct>();
    await products.ParallelForEachAsync(
        async product =>
        {
            OsmData osmData;
            try
            {
                osmData = await GetOsmData(_osmUrl.Replace("articlenumber",
                    product.MaterialNumber.ToString())).ConfigureAwait(false);
            }
            catch (Exception e)
            {
                osmData = null;
            }

            if (osmData != null && osmData.PrintingData.Count > 0)
            {
                product.OsmPrintImageAvailable = true;
            }
            else
            {
                product.OsmPrintImageAvailable = false;
            }

            result.Add(product);
        },
        // 0 => Chooses a default value based on the processor count
        maxDegreeOfParallelism: 0
        );
    return result;
}

You shouldn't iterate over a collection while you're modifying it. If you want to keep a "queue of work" to do, then you can use something like TPL Dataflow. Particularly since I'm not at all clear how async streams **or** parallelism is useful here. — Stephen Cleary, Apr 29 '19 at 15:14
@StephenCleary thanks for your response! I think i dont modify the collection (adding or deleting elements) i just set a property of the elements, I dont get where this should be a problem? I have to do web requests for 3000 to 5000 products, is that not a good place for parallelism? — Marvin, Apr 30 '19 at 11:08
Ah yes, I misread `result.Add` as modifying the collection, but that's a different collection. If you have multiple web requests to do, consider asynchronous concurrency (`Task.WhenAll`). Parallelism uses multiple threads, which you don't need AFAICS. — Stephen Cleary, Apr 30 '19 at 14:22

score 0 · Accepted Answer · answered Apr 30 '19 at 08:59

With the help of my colleague I could solve the problem... The problem was not inside the method itself, but only how it was called. I called it form the Main/UI-Thread synchronously. That seems to have caused a deadlock. Making the calling method async and awaiting the CheckOsmDataAsync() solved the problem.

Nevertheless thanks for your responds!

Why does this ParallelForEachAsync method never return?

1 Answers1