1

I 've the following code:

int counter = 1;
var lockTarget = new object();
Parallel.For(1, totalSIM,  i => {

    /* do some stuff  */

    lock(lockTarget) {
        _printConcurrent(counter++);
    }

});

and I need to call _printConcurrent every ~200 times, not every time. I thought about do a MOD (%) for counter, but I can't understand how to use the condition. I enclosed the lock in an if but I can't use the counter so I'm in a logic loop instead the code :S

Any help will be appreciated.

_printConcurrent is only an output of the process, I sacrifice some performance for monitoring data.

Theodor Zoulias
  • 34,835
  • 7
  • 69
  • 104
Leandro Bardelli
  • 10,561
  • 15
  • 79
  • 116

2 Answers2

2

You could increment atomically the counter without locking, with the Interlocked.Increment method, and enter the lock only if the result of the atomic increment is divisible by 200:

int counter = 0;
ParallelOptions options = new() { MaxDegreeOfParallelism = Environment.ProcessorCount };
Parallel.For(0, totalSIM, options, i => {

    /* do some stuff */

    int current = Interlocked.Increment(ref counter);
    if (current % 200 == 0)
    {
        lock (options) _printConcurrent(current);
    }

});

This way the _printConcurrent will be called once every 200 iterations, without blocking the rest 199 iterations.

Update: The above approach has a minor flaw. It doesn't guarantee that the _printConcurrent will be called sequentially with an incremented argument. For example it is theoretically possible that the _printConcurrent(400) will be called before the _printConcurrent(200). That's because the operating system can suspend any thread at any time for a duration of around 10-30 milliseconds (demo), so a thread could be suspended immediately after the Interlocked.Increment line, and lose the race to enter the lock by a non-suspended thread. In case this is a problem, you can solve it be throwing a second counter in the mix, that is incremented only inside the lock:

int counter1 = 0;
int counter2 = 0;
const int step = 200;
ParallelOptions options = new() { MaxDegreeOfParallelism = Environment.ProcessorCount };
Parallel.For(0, totalSIM, options, i => {

    /* do some stuff */

    if (Interlocked.Increment(ref counter1) % step == 0)
    {
        lock (options) _printConcurrent(counter2 += step);
    }

});
Theodor Zoulias
  • 34,835
  • 7
  • 69
  • 104
  • 1
    wow, I never thought use both, I refactor the Interlocked idea in order to call the method. Thanks a lot man! Sometimes the simplest ideas are the hardest to find and the most powerful. – Leandro Bardelli Mar 09 '23 at 04:35
  • 1
    Thanks for this also! ParallelOptions options = new() { MaxDegreeOfParallelism = Environment.ProcessorCount }; I didnt know it. – Leandro Bardelli Mar 09 '23 at 04:37
  • 1
    @LeandroBardelli yep, I am a big fan of specifying the [`MaxDegreeOfParallelism`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.paralleloptions.maxdegreeofparallelism) explicitly in every use of the `Parallel` class! – Theodor Zoulias Mar 09 '23 at 04:41
  • Thanks a lot! I will try this also! By the way, I think the answer is "no" but there is a way to do this without lock it? In order to "accelerate" the loop? (Im learning multithreading) – Leandro Bardelli Mar 09 '23 at 12:47
  • That's PERFECT ! Thanks a lot!!! I had the doubt about Interlocked – Leandro Bardelli Mar 09 '23 at 12:58
  • @LeandroBardelli another approach on reporting progress is to have a different thread ([or asynchronous execution flow](https://stackoverflow.com/questions/30462079/run-async-method-regularly-with-specified-interval "Run async method regularly with specified interval")) querying periodically the `counter`, so that the `Parallel` worker threads don't have to coordinate and block each other for serializing the progress reporting work. They'll be free to dedicate themselves to the actual parallel work. You could try it, but don't expect major performance improvements. – Theodor Zoulias Mar 09 '23 at 15:06
  • 1
    @LeandroBardelli in case the `/* do some stuff */` is a very lightweight operation (less than a microsecond), you could improve the overall performance by partitioning the range `0, totalSIM` using the `Partitioner` class, and processing a range of numbers in each invocation of the `body` instead of a single number. See [here](https://stackoverflow.com/questions/70839214/adding-two-arrays-in-parallel/70840104#70840104) for an example. – Theodor Zoulias Mar 09 '23 at 15:20
  • 1
    perfect!!!! yes, I discover that in some times "some stuff" is so simple that Parallel.For can even work more slowly than a "normal for" and in other cases (idk why, maybe I will ask here) the mod % is the reason why everything goes slow. Il try this logic! Thanks a lot for everytihng Im learning A LOT – Leandro Bardelli Mar 09 '23 at 15:39
1

I do like the Rx or System.Reactive approach for doing this. It's quite neat.

Observable
    .Range(0, totalSIM)
    .Select(x => Observable.Start(() => { /* do stuff */ }))
    .Merge(Environment.ProcessorCount)
    .Select((_, n) => n + 1)
    .Where(x => x % 200 == 0)
    .Subscribe(n => _printConcurrent(n));
Enigmativity
  • 113,464
  • 11
  • 89
  • 172
  • Thanks! I used Observable in an angularJS project and I totally forget it , I'll try it because I'm looking for a way to accelerate a bucle that made some math simulations – Leandro Bardelli Mar 09 '23 at 12:48
  • 1
    It's neat indeed, but Rx doesn't behave well in case of exceptions (it [abandons fire-and-forget work](https://stackoverflow.com/questions/70901276/how-to-implement-a-custom-selectmany-operator-that-waits-for-all-observable-subs) when the first exception occurs). Also in this particular example the operation is not waited or awaited, and any exception will be unhandled and will cause the process to crash. Using Rx for the purpose of parallelization is a stress for this technology. It wasn't invented to solve this problem. – Theodor Zoulias Mar 09 '23 at 14:58
  • I really try to code to avoid exceptions and to get my code to a state that exceptions are truly exceptional - and are unhandlable, like running our of hard drive space or memory. Doing that tends to make this kind of thing work. – Enigmativity Mar 09 '23 at 22:04
  • How are you doing this? By putting every call in a `try`/`catch`, and inside the `catch` you log the error and return `default`? If so, you could demonstrate this approach in the answer itself. Assume that both the `/* do stuff */` and the `_printConcurrent` can throw [exogenous](https://ericlippert.com/2008/09/10/vexing-exceptions/) exceptions, and get the code to a state that only fatal exceptions can escape. – Theodor Zoulias Mar 09 '23 at 22:54
  • @TheodorZoulias - I do much like the example in Eric Lippert's blog post you referenced. I try to avoid `try`/`catch` where possible and anticipate the errors that can occur - I live with the fact that there might now be race conditions, but I've reduced the likelihood that the error can occur. Then, if necessary, I return a `Maybe` or an `Exceptional`, rather than `T`, that allows the query to progress without an error mucking things up. – Enigmativity Mar 09 '23 at 23:06
  • Well, then, why don't you demonstrate that approach, and instead you are posting code that crashes the process in case of an error? That approach (the `Maybe`/`Exceptional` approach) is likely to have other problems, but in order to discuss that problems we need to see it first! – Theodor Zoulias Mar 09 '23 at 23:45
  • Invite me to Athens and perhaps we can do some pair coding. That'll be easier than coming up with something non-trivial here. – Enigmativity Mar 10 '23 at 05:03
  • Ha ha! If you ever come to Athens you'll have better things to do than pair coding, or demonstrating railway-oriented programming! :-) – Theodor Zoulias Mar 10 '23 at 07:31