0

I've got a worker that processes a cloud queue messages. For each queue message, it creates a task and runs some short lambda with 2 IO operations.

For example, this is the lambda:

  1. Deserialize queue message's payload
  2. Send REST request to endpoint x (non-blocking IO operation)
  3. Do small computation work (sort of JsonConvert.Deserialize)
  4. Enqueue new message to another cloud queue (non-blocking IO operation)

The worker is configured to keep up to 2000 running tasks.

Running this code solely on a VM - I can see that there are 14K Context Switches / sec and there are 300-500 threads running.

I'm wondering how many threads run in the default TaskScheduler thread pool.

How can I check that?

johni
  • 5,342
  • 6
  • 42
  • 70
  • What is your real question? What will you solve with that information? – Sir Rufo Jun 17 '17 at 17:48
  • My thought was that if there are many threads running in the pool (tens or hundreds) - they produce a lot of context switches - and if that's the case.. I will use a custom TaskScheduler with constant amount of threads in the pool. – johni Jun 17 '17 at 17:52
  • The threads in the pool are alive but not all are working. Some are just waiting for work/termination. You should think about how you organize your operations. TPL DataFlow will be the right starting point – Sir Rufo Jun 17 '17 at 18:00
  • How come a thread in that pool will wait for work/termination? they execute the first sync part of the code, and execute a non-blocking IO operation, until the IO thread of the OS sends the event. What am I missing? – johni Jun 17 '17 at 18:03
  • Because it is a pool? Creating a thread is expensive and the pool will keep the threads alive for reusing or kick them off, after an amount of time – Sir Rufo Jun 17 '17 at 18:07
  • Ok. We went out of scope. I was asking something very specific. Is it possible to determine, in any given moment, how many threads does that pool hold? – johni Jun 17 '17 at 18:11
  • AFAIK you can't – Sir Rufo Jun 17 '17 at 18:13
  • 2
    The right way is not building a custom Pool/Scheduler but using TPL DataFlow – Sir Rufo Jun 17 '17 at 18:15
  • Wasn't I right to assume that _in case_ that pool had 200+ threads running, then using `LimitedConcurrencyLevelTaskScheduler` with concurrency of 2x #cores - would save a lot of Context Switches? – johni Jun 17 '17 at 18:15
  • Why would you want to do that? It will affect the whole process (global) and you want to solve a local problem. With DataFlow you can shape your operations for each step and limit the degree of parallelism for each. Much better than that big hammer slamming on all – Sir Rufo Jun 17 '17 at 18:18
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/146956/discussion-between-johni-and-sir-rufo). – johni Jun 17 '17 at 18:18
  • Don't confuse threads and tasks. Also, if you use `ConfigureAwait(false)`, there will be less number of context switches – VMAtm Jun 17 '17 at 22:43
  • Use where? In every task created in the lambda that is used in the the Task.Factory.Startnew? – johni Jun 17 '17 at 23:09
  • @johni Keep in mind that the threadpool uses a feedback-loop mechanism to tune the number of active threads. While there are some corner-cases, most of the time it does a fine job. Before trying to optimize the context-switches, make sure you will actually gain something from it. A context-switch is roughly 5µs. Even doing 10.000 of them, it's only 5 ms, across all the CPU cores (so if you have 4 cores, that's less than 2 ms per core). Do you really have an application that could benefit from a 2ms/second increase in latency? – Kevin Gosse Jun 19 '17 at 06:22
  • @KevinGosse that's a very good point. I wasn't aware that it takes 5'u's. how can I measure that? – johni Jun 19 '17 at 06:30
  • This is really hard to measure. 5µs is really a ballpark estimation: some tests report as low as 1µs, some others as high as 10µs (https://stackoverflow.com/a/22421360/869621, http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-context.html, https://blogs.msdn.microsoft.com/andrewarnottms/2012/12/28/the-cost-of-context-switches/). – Kevin Gosse Jun 19 '17 at 06:55
  • I suggest that you run a profiler on your code (the timeline view in JetBrains dotTrace is excellent, and you get 10 days free trial) so you understand exactly what's taking time. It won't tell you how much time you spend in context-switch, but you'll get a feel of how much CPU the user code + task infrastructure + GC are taking, and can naturally deduce that context-switching doesn't cost much in comparison – Kevin Gosse Jun 19 '17 at 06:56

0 Answers0