Limit number of Threads without waiting for them

Question

I have a foreach loop that sends data to a GRPC API. I want my loop to send multiple requests at the same time but limit the number of requests to e.g. 10. My current code is the following:

foreach (var element in elements)
{
     var x = new Thread(() =>
        SendOverGrpc(element));
     x.Start();
}

But with that code, the software "immediately" sends all requests. How can I limit the number of requests to e.g. 10? As soon as one of my 10 requests is finished, I want to send the next one.

You don't need to use low-level threading to achieve concurrent requests in network IO (the main reason being [it's incredibly wasteful](https://stackoverflow.com/questions/9964899/why-are-os-threads-considered-expensive): doing `new Thread` consumes a big chunk of RAM for the stack and thread-local storage, which is why .NET comes with the `ThreadPool`). Instead you should use proper async sockets which enables programs to achieve concurrent request processing very efficiently. — Dai, Feb 15 '23 at 22:28

score 2 · Accepted Answer · answered Feb 15 '23 at 22:23

2

Easiest way is Parallel.Foreach, eg

Parallel.ForEach(elements, new ParallelOptions() { MaxDegreeOfParallelism = 10 }, element  =>
    {
      SendOverGrpc(element);
    });

answered Feb 15 '23 at 22:23

David Browne - Microsoft

80,331
6
39
67

This is bad advice though: threads should not be used as the primary means of multiplexing network IO, even when used via `Parallel.ForEach`. Async IO does not require multiple threads and using that instead of Thread-based approaches avoids numerous bugs relating to cross-thread synchronization and race-conditions. – Dai Feb 15 '23 at 22:27
Using 10 threads from the thread pool is trivial, as long as there aren't a many concurrent invocations of this method. – David Browne - Microsoft Feb 15 '23 at 22:31
@Dai can you please elaborate regarding "Async IO", are you sure you aren't confusing parallel versus asynchronous? – OlegI Feb 15 '23 at 22:33
@Dai For grpc it might indeed be a bad advice, however according to the title this would be a valid answer. Perhaps you could post a link about async IO+Grpc. I agree that it's something that needs to be taken into account – Pieterjan Feb 15 '23 at 22:38
@OlegI True parallelism in network IO just isn't a thing (unless you have a multihomed network adapter, I suppose). While [asynchronous, non-blocking, sockets](https://stackoverflow.com/questions/10654286/why-should-i-use-non-blocking-or-blocking-sockets) make it possible to effectively _fire-off_ many network requests/messages/packets at once using only a single application/userland program thread and let the OS's network stack handle any concurrency for you. – Dai Feb 15 '23 at 22:41
@Pieterjan gRPC doesn't dictate any particular client-library nor require client-libraries to support language-native asynchronous patterns (like TPL in .NET), so it isn't possible to give a general answer to a general topic like that. w.r.t. .NET in particular, I've seen good - and awful - gRPC client code, and I don't know what gRPC client/proxy codegen or tooling the OP is using so I can't say. – Dai Feb 15 '23 at 22:43
Also answers aren’t advice. – David Browne - Microsoft Feb 15 '23 at 23:17
@DavidBrowne-Microsoft _"answers aren’t advice"_ - **Que?** – Dai Feb 16 '23 at 20:02

Limit number of Threads without waiting for them

1 Answers1