When are ThreadPool threads are released back to the ThreadPool?

Question

Can anyone explain (or have a resource that explains) exactly when ThreadPool threads are released back to the ThreadPool? Here is a small example program (Dotnet fiddle: https://dotnetfiddle.net/XRso3q)

    public static async Task Main()
    {
        Console.WriteLine("Start");
        var t1 = ShortWork("SW1");
        var t2 = ShortWork("SW2");
        await Task.Delay(50);
        var t3 = LongWork("LW1");
        Console.WriteLine($"After starting LongWork Thread={Thread.CurrentThread.ManagedThreadId}");
        await Task.WhenAll(t1, t2);
        await t3;
        Console.WriteLine("Done");
    }
    
    public static async Task ShortWork(string name)
    {
        Console.WriteLine($"SHORT Start {name} Thread={Thread.CurrentThread.ManagedThreadId}");
        await Task.Delay(500);
        Console.WriteLine($"SHORT End {name} Thread={Thread.CurrentThread.ManagedThreadId}");
    }
    
    public static async Task LongWork(string name)
    {
        Console.WriteLine($"LONG Start {name} Thread={Thread.CurrentThread.ManagedThreadId}");
        await Task.Delay(2500);
        Console.WriteLine($"LONG End {name} Thread={Thread.CurrentThread.ManagedThreadId}");
    }

Outputs:

Start
SHORT Start SW1 Thread=1
SHORT Start SW2 Thread=1
LONG Start LW1 Thread=5
After starting LongWork Thread=5
SHORT End SW1 Thread=7
SHORT End SW2 Thread=5
LONG End LW1 Thread=5
Done

Long work starts on thread 5, but at some point thread 5 is released back to the threadpool as thread 5 is able to pick up Short SW1 ending. When exactly does 5 get released back to the threadpool after await Task.Delay(2500) in LongWork? Does the await call release it back to the threadpool? I dont think this is the case as if I log the thread id right after the call to LongWork, that is still running on thread 5. await Task.WhenAll is called on thread 5 - which then releases control back up to whatever called 'main' - is this where it gets released as there is no 'caller' to go back to?

My understanding of what happens:

• Starts on thread 1, thread 1 executes ShortWork SW1 and SW2.
• Task.Delay(50) is awaited and thread 1 gets released (as there is no more work to do?)
• Thread 5 is chosen to pick up the continuation after the 50ms delay
• Thread 5 kicks off LongWork, and it it gets to the awaited 2500ms delay. Control gets released back up to main, still on thread 5. t1 and t2 are awaited - control gets released back up to whatever called main (and so thread 5's work is done - it gets released to the threadpool)
• At this point no threads are 'doing' anything
• When the ShortWork delay is done, thread 5 and 7 are selected from the pool for the continuation of each call. Once done with the continuation, these are released to the pool (?)
• Another thread picks up the continuation between Task.WhenAll and await t3, which then immediately gets released because it is just awaiting t3
• A ThreadPool thread is selected to do the continuation of the LongWork call
• Finally, a ThreadPool thread picks up the last work to write done after t3 is done.

Also as a bonus, why does 5 pick up end of SW1, and 7 pick up end of LW1? These are the threads that were just used. Are they somehow kept as 'hot threads and prioritised for continuations that come up?

Answer 1

The way await works is that it first checks its awaitable argument; if it has already completed, then the async method continues synchronously. If it is not complete, then the method returns to its caller.

A second key to understanding is that all async methods begin executing synchronously, on a normal call stack just like any other method.

The third useful piece of information here is that a Console app needs a foreground thread to keep running or it will exit. So when you have an async Main, behind the scenes the runtime blocks the main thread on the returned task. So, in your example, when the first await is hit in Main, it returns a task, and the main thread 1 spends the rest of the time blocked on that task.

In this code, all continuations are run by thread pool threads. It's not specified or guaranteed which thread(s) will run which continuations.

The current implementation uses synchronous continuations, so in your example the thread id for LONG Start LW1 and After starting LongWork will always be the same. You can even place a breakpoint on After starting LongWork and see how a LongWork continuation is actually in the call stack of your Main continuation.

Answer 2

What actually happens is that when a pool thread starts, it takes a task from the pool's work queue, waiting if necessary. It then executes the task, and then takes a new one, again waiting if necessary. This is repeated until the thread determines that it has to die and exits.

While the thread is taking a new task, or waiting for one, you could say that it has been "released to the thread pool", but that's not really useful or helpful. Nothing from the pool actually does things to the threads, after starting them up. The threads control themselves.

When you write an async function, the compiler transforms it in a way that divides it into many tasks, each of which will be executed by a function call. Where you write Task.Delay(), what actually happens is that your function schedules the task that represents the remainder of its execution, and then returns all the way out to the thread's main procedure, allowing it to get a new task from the thread pool's work queue.