Run async task continuation on native function callback thread

Question

I have a C function FsReadStream that does some asynchronous work and takes a callback. When done, it calls the callback using the QueueUserWorkItem windows function.

I am trying to call this function from managed code (c#) using the async/await pattern. So I do the following

1. Construct a Task object passing the constructor a lambda that returns the result.
2. Construct a callback that runs this task using the RunSynchronously method
3. Call the asynchronous native function, passing in the callback
4. Return the task object to the caller

My code looks something like this

/// Reads into the buffer as many bytes as the buffer size
public Task<ReadResult> ReadAsync(byte[] buffer)
{
    GCHandle pinnedBuffer = GCHandle.Alloc(buffer, GCHandleType.Pinned);
    IntPtr bytesToRead = Marshal.AllocHGlobal(sizeof(long));
    Marshal.WriteInt64(bytesToRead, buffer.Length);

    FsAsyncInfo asyncInfo = new FsAsyncInfo();
    ReadResult readResult = new ReadResult();

    Task<ReadResult> readCompletionTask = new Task<ReadResult>(() => { return readResult; });
    TaskScheduler scheduler = TaskScheduler.FromCurrentSynchronizationContext();

    asyncInfo.Callback = (int status) =>
    {
        readResult.ErrorCode = status;
        readResult.BytesRead = (int)Marshal.ReadInt64(bytesToRead);
        readCompletionTask.RunSynchronously(scheduler);
        pinnedBuffer.Free();
        Marshal.FreeHGlobal(bytesToRead);
    };

    // Call asynchronous native method    
    NativeMethods.FsReadStream(
                    pinnedBuffer.AddrOfPinnedObject(),
                    bytesToRead,
                    ref asyncInfo);

    return readCompletionTask;
}

and I call it like this

ReadResult readResult = await ReadAsync(data);

I have two questions

1. How to make the code that runs after the call to await ReadAsync run on the same thread as the callback? Currently, I see it runs on a different thread even though I am calling readCompletionTask.RunSynchronously. I am running this code under ASP.NET and IIS.
2. Does the native QueueUserWorkItem function use the same threadpool as the managed ThreadPool.QueueUserWorkItem method? My opinion was that it should, and therefore it should be possible for the managed TaskScheduler to schedule tasks on the native callback thread.

Answer 1

You shouldn't use the Task constructor in modern code. At all. Ever. There is no use case for it.

In this case, you should use TaskCompletionSource<T>.

How to make the code that runs after the call to await ReadAsync run on the same thread as the callback?

You can't guarantee it; await just doesn't work that way. If the code absolutely must execute on the same thread, then it should be called directly from the callback.

However, if it's just preferred to execute on the same thread, then you don't have to do anything special; await already uses the ExecuteSynchronously flag:

public Task<ReadResult> ReadAsync(byte[] buffer)
{
  var tcs = new TaskCompletionSource<ReadResult>();
  GCHandle pinnedBuffer = GCHandle.Alloc(buffer, GCHandleType.Pinned);

  IntPtr bytesToRead = Marshal.AllocHGlobal(sizeof(long));
  Marshal.WriteInt64(bytesToRead, buffer.Length);

  FsAsyncInfo asyncInfo = new FsAsyncInfo();
  asyncInfo.Callback = (int status) =>
  {
    tcs.TrySetResult(new ReadResult
    {
      ErrorCode = status;
      BytesRead = (int)Marshal.ReadInt64(bytesToRead);
    });
    pinnedBuffer.Free();
    Marshal.FreeHGlobal(bytesToRead);
  };

  NativeMethods.FsReadStream(pinnedBuffer.AddrOfPinnedObject(), bytesToRead, ref asyncInfo);

  return tcs.Task;
}

Does the native QueueUserWorkItem function use the same threadpool as the managed ThreadPool.QueueUserWorkItem method?

No. Those are two completely different thread pools.

Answer 2

How to make the code that runs after the call to await ReadAsync run on the same thread as the callback?

This is not possible in a reliable way. ExecuteSynchronously is not a guarantee. RunSynchronously does not guarantee it either. You can of course pass in a callback and call that callback synchronously.

Also, what does FromCurrentSynchronizationContext return? My spider sense is telling me that this is based on a misunderstanding...

Does the native QueueUserWorkItem function use the same threadpool as the managed ThreadPool.QueueUserWorkItem method?

I don't think so and even if this was the case you could not target a particular thread. You only could target a particular pool.

Why do you need to execute on the same thread? Usually, people who ask about that really do want and need something else.

---

Your way to create and return a task is very odd. Why are you not using the standard pattern based on TaskCompletionSource?

---

I think you have a GC hole because nothing keeps asyncInfo.Callback alive. It can be collected away while the native call is in progress. Use GC.KeepAlive in the callback.