1

Good afternoon, dear forum users! I have used PLINQ a lot and have never had a problem with it until now. I am writing a universal logger to use in several projects that could write logs simultaneously in several ways (files, mail, database, http requests) and in several sources for each method. Accordingly, I need to use concurrency and locks for writing to files.On tests found my code freezes when using AsParallel.ForAll and works perfectly fine for Parallel.ForEach.

The calling code looks like this:

var loggers = new List<ILogger>() { logger1, logger2, logger3, logger4, logger5, logger6 };
var message = "Hello world";

Parallel.For(0, 10000, e =>
{
    loggers.AsParallel().ForAll(e =>
    {
         e.Write(message);
    });
});

This piece of code seems to work fine. However the piece below which is the implementation of the Write () call only works with Parallel.ForEach. This code works fine:

public void Write(string message)
{
    Task.Run( () =>
    {
        Parallel.ForEach(activeLoggers, log =>
        {
            log.Write(message);
        });

     });
}

This code freezes and does not reach the Write () method at all:

public void Write(string message)
{
    Task.Run( () =>
    {
        activeLoggers.AsParallel().ForAll(e =>
        {
            e.Write(message);
        });
    });
}

In this context, activeLoggers is an ordinary list of loggers, each of which has its own sources for recording. It probably makes no sense to give an example of their implementation, since the program freezes until it is called in the second case. I would like to use a program in Asp Net Core 3.1, in the calling method logger1, logger2, etc. are located in different controllers, but can access the same sources (for example, files). I looked for a solution to the problem in the following topics, but unfortunately I couldn't find one:

AsParallel.ForAll vs Parallel.ForEach

Parallelism in .NET – Part 8, PLINQ’s ForAll Method

Parallel.ForEach vs AsParallel().ForAll

s AsParallel().ForAll reliable

The project is a library on Net Standard 2.0, tested on Net Core 3.1. Anyone have any ideas what the problem might be?

Slava
  • 27
  • 2
  • I will add that the activeLoggers list is of type IBaseLogger, not ILogger. It would probably be more correct to name it "Executor" – Slava Aug 16 '20 at 03:20
  • 1
    Your first code sample, using `AsParallel` nested inside a `Parallel.For` loop, over-parallelizes your workload. This could cause `ThreadPool` starvation in application types with small initial `ThreadPool` size, like WinForms/WPF. I suggest to parallelize either the outer or the inner loop, not both. – Theodor Zoulias Aug 16 '20 at 04:28
  • Regarding the second and third code sample, what is the type of the `activeLoggers`? Is it `List`? Also have you tried if the observed difference in behavior persists if you run the same code in a different application type, like a Console app? I am asking because doing [fire-and-forget in ASP.NET](https://blog.stephencleary.com/2014/06/fire-and-forget-on-asp-net.html) is a well known bad practice, with non-deterministic outcome. – Theodor Zoulias Aug 16 '20 at 04:37
  • @TheodorZoulias, Thanks for your reply. I've only tested methods in a console application and in unit tests. I am writing it for Asp net core, but never run any code there. The type for `activeLoggers` is `List ` . This type is an abstraction over several logging mechanisms. Thus, the classes (TextLog, HttpLog, MailLog, DatabaseLog) are inherited from IBaseLogger. In each of them there are sources to which the record is kept. So `TextLog` can write to several text files, and `MailLog` can send messages to several mail addresses with different smtp settings. – Slava Aug 16 '20 at 07:59
  • @TheodorZoulias, Sorry about testing on Asp Net Core, unfortunately I wrote it wrong. English is not my native language. – Slava Aug 16 '20 at 08:04
  • That code shows Parallel -> Parallel -> Parallel, how many cores do you have in your CPU? Parallelizing so much is only going to make everything extremely slow. Instead of doing all this useless parallelization, use an event-based system and let each logger implementation log whenever they are able to – Camilo Terevinto Aug 16 '20 at 08:45
  • 1
    I see no reason why the `AsParallel().ForAll` should behave much differently than `Parallel.ForEach`. My guess is that the difference in behavior is probably caused by code that you have not included in the question. Also IMHO the system you are trying to make is inherently non-robust, since it is based on fire-and-forget tasks, so finding the cause of the discrepancy has probably only academic value. Instead of wasting your time fixing this, you could try reworking your architecture by using something queue-based like a `BlockingCollection` or a `Channel` or the TPL Dataflow. – Theodor Zoulias Aug 16 '20 at 09:26
  • 1
    @TheodorZoulias According to MSDN, there is some difference between `AsParallel().ForAll` and `Parallel.ForEach`, however it's not explained very well: https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/potential-pitfalls-with-plinq#prefer-forall-to-foreach-when-it-is-possible I do agree that the problem is probably unrelated though, at least not directly. Perhaps PLINQ is smart enough to run the tasks on a single thread and avoids some deadlock? – asaf92 Aug 17 '20 at 12:11
  • @asaf92 indeed, the `AsParallel().ForAll` and `Parallel.ForEach` do not have identical implementations, but there is no apparent reason why the one would work and the other would freeze/deadlock in OP's code. Both use by default all the available cores of the machine, and both use the current thread as one of the worker threads. – Theodor Zoulias Aug 17 '20 at 12:18
  • Thanks everyone for the answers. I have not found an answer to my question. But the option with queues seems to me the most attractive, despite the fact that it implies one constantly clogged thread. At least I have no problem with him. Answering one of the questions: there are 32 cores on the machine. – Slava Feb 23 '21 at 14:21

0 Answers0