1

When using a multi-sender/multi-receiver channels like crossbeam-channel or async-channel, I would like to dynamically scale the number of receivers based on the channel congestion state. If there are too many pending item is the channel, more receivers should be created. If the channel is near-empty, some receivers may need to stop.

Is there an API in either of the popular multi-receiver channel implementations that would help with this? Is there a different approach for this?

Yuri Astrakhan
  • 8,808
  • 6
  • 63
  • 97
  • 2
    By "receivers" do you mean the number of threads receiving? Or the number of asynchronous tasks doing that? Or what? If you mean threads, don't do it, just spawn a thread or 2 for every core, spawning more will not improve your throughput. A sleeping thread isn't going to use much resources at all. – cafce25 Aug 19 '23 at 05:34
  • Receivers are file sinks, each writing received data to their own file, and rotating those files once their size exceed some size. I do not want to have too many pending semi-empty files. Instead, I would like all files to be actively being written to, but only as many parallel writing as needed by current load. – Yuri Astrakhan Aug 20 '23 at 15:40

0 Answers0