Run ValueTasks on a custom thread pool

Question

I am looking to execute a bunch of ValueTask-returning functions on a custom thread pool - i.e. on a bunch of threads I'm spawning and handling myself, rather than the default ThreadPool.

Meaning, all synchronous bits of these functions, including any potential task continuations, should be executed on my custom thread pool.

Conceptually, something like:

class Example
{
    async ValueTask DoStuff(int something)
    {
        // .. do some stuff in here, might complete synchronously or not, who knows ..
    }

    private void Test()
    {
        for (int i = 0; i < 1_000; i++)
        {
            Func<ValueTask> method = () => DoStuff(1);
            MyThreadPool.Queue(method);
        }
    }
}

What's the best way to do this?

My current approach is something like this:

class Example
{
    async ValueTask DoStuff(int something)
    {
        // .. do some stuff in here, might complete synchronously or not, who knows ..
    }

    private void Test()
    {
        SynchronizationContext myContext = new MyCustomThreadPoolSynchronisationContext();
        TaskScheduler myScheduler;
        
        var prevCtx = SynchronizationContext.Current;
        try
        {
            SynchronizationContext.SetSynchronizationContext(myContext);
            myScheduler = TaskScheduler.FromCurrentSynchronizationContext();
        }
        finally
        {
            SynchronizationContext.SetSynchronizationContext(prevCtx);
        }

        var myTaskFactory = new TaskFactory(myScheduler);
        
        
        for (int i = 0; i < 1_000; i++)
        {
            myTaskFactory.StartNew(() => DoStuff(i).AsTask());
        }
    }
}

This seems to work, but having to convert the ValueTask into a Task and lodging it to a TaskFactory feels exceptionally clunky. And having to install my synchronization context, just to be able to get a hold off an appropriate TaskScheduler (and then immediately falling back onto the old synchronization context) feels quite off too.

Are there any conceptual flaws with my current approach?

Better yet, are there any better, less awkward ways of doing this?

I fail to understand the awesomeness of a custom thread pool compared to a bunch of *I don't care who executes them* Tasks... so... **WHY?** — grek40, Sep 17 '21 at 13:11
Better resource control. Imagine a huge dependency graph that hogs as much (CPU) resource as you let it during calculations. If everything just lands up on the thread pool, this work might stuff up everything and leave little time (or little predictability) for other types of work in the application, or even in the system. Better to say: Machine has X threads, we'll dedicate some fraction of those work for of type A and others for work of type B. (Plus neat little side benefits; potentialy cache coherence, easier to debug if you don't fish in gazillion "Background threads", etc.) — Bogey, Sep 17 '21 at 13:26
So, you're taking a whole load of methods that have essentially been declared as "I can often complete synchronously with no thread switching or allocations" and you're forcing them onto another thread. Why? — Damien_The_Unbeliever, Sep 17 '21 at 13:40
@Damien_The_Unbeliever They may potentially complete synchronously, but not necessarily fast. Many of them will be doing CPU-heavy work (yet have no dependencies on each other, i.e. can be parallelized). I hence need to offload them to a bunch of calculation threads. (But I don't just want want to lodge potentially thousands of CPU-intense maybe-or-maybenot-asynchronous things onto the shared threadpool, that might completely stuff it up for anything else) — Bogey, Sep 17 '21 at 14:25
Why don't you use a TaskScheduler that is limited to run on x threads of the shared threadpool? Rather than a custom threadpool. — Rowan Smith, Sep 17 '21 at 14:27
A custom thread pool is one of the worst ideas I've ever heard. — Liam, Sep 17 '21 at 14:42
Is [this](https://stackoverflow.com/questions/60899507/execute-certain-background-tasks-in-separate-threadpool-to-avoid-starvation-to-c/60913417#60913417) `RunLowPriority` idea any useful? Btw why have you opted for `ValueTask`s instead of `Task`s? For avoiding the allocation of `Task` objects? These objects are pretty small. How many of them do you anticipate creating per second? Unless the number has five digits or more, it's unlikely that the benefits will justify the pain of having to deal with `IValueTaskSource`s and `ManualResetValueTaskSourceCore`s. — Theodor Zoulias, Sep 17 '21 at 14:46
@RowanSmith If there is some out-of-the-box scheduler that does this, it would certainly be worth considering! (If there isn't and I need to write it up myself anyway, I'd rather stick to my custom thread pool for benefits of naming etc.)! Either way though I would presumably still be facing the question of how to schedule these ValueTasks on it efficiently — Bogey, Sep 17 '21 at 14:48
Thanks @TheodorZoulias , very interesting approach. In an ideal world I still prefer a customized thread pool, but its a pain to implement efficiently with work stealing queues and what not. This might be a good trade-off! I would expect up to a few thousand (Value)Tasks per second at peak. The rest of the app is quite heavy on the GC by nature of its work, always nice to reduce pressure a little where possible/sensible - no dealbreaker though. Just felt odd there didn't seem to be any good way (i could think of) to do with ValueTasks — Bogey, Sep 17 '21 at 14:56
@Bogey there seems to be a misunderstanding of how TPL in general works. Resources and threads are *not* controlled by the thread pool. They're controlled by the TaskScheduler. The tasks themselves, whether `Task` or `ValueTask` know nothing about either the TaskScheduler or the thread pool. If you want to limit how many tasks can execute at a time, you use eg the [LimitedConcurrencyTaskScheduler](https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.taskscheduler?view=net-5.0#examples) in the TaskScheduler example. — Panagiotis Kanavos, Sep 17 '21 at 15:08
You could check out [this](https://stackoverflow.com/questions/69147931/avoiding-allocations-and-maintaining-concurrency-when-wrapping-a-callback-based/) question, to get an idea about what a zero-allocations `ValueTask.Run` implementation might look like. You will have to manage a pool of `IValueTaskSource` objects, take decisions about the size of this pool, learn what's the correct moment that an object can be safely returned to the pool etc. It's a serious undertaking. And then you'll have to live with the `ValueTask` limitations (`await` only once etc). — Theodor Zoulias, Sep 17 '21 at 15:16
Since you expect CPU heavy tasks, maybe `TaskFactory.StartNew` with `TaskCreationOptions.LongRunning` would be a better option than a new scheduler... this option looks like it is designed to prevent task starvation due to long running tasks blocking the threads in the pool. — grek40, Sep 21 '21 at 20:41

Rowan Smith · Accepted Answer · 2021-09-17T14:58:46.997

Everything I've read about creating a custom ThreadPool says don't.

An alternative would be to use a custom TaskScheduler on the shared ThreadPool.

You can use this TaskScheduler class like this:

static async Task Main(string[] args)
{
    // Create a scheduler that uses four threads.
    LimitedConcurrencyLevelTaskScheduler lcts = new LimitedConcurrencyLevelTaskScheduler(4);
    List<Task> tasks = new List<Task>();

    TaskFactory factory = new TaskFactory(lcts);
    CancellationTokenSource cts = new CancellationTokenSource(10000);

    // Start 20 tasks that will run 4 threads at 100% CPU
    for (var i = 0; i < 20; i++)
        tasks.Add(factory.StartNew(() => {
            while (true)
                if (cts.Token.IsCancellationRequested)
                    break;
        },cts.Token));

    await Task.WhenAll(tasks);
    cts.Dispose();
}

Run ValueTasks on a custom thread pool

1 Answers1