Asynchronous output redirection in Win32

Question

This is about launching a child process and redirecting output.

In .NET, we could use Process.RedirectStandard... to redirect the standard IO, and use Process.OutputDataReceived to receive an event when something was written to the output.

I want to implement this in native C++ with Win32 API.

As I looked into the source (1, 2), it looks like it's using Stream.BeginRead method, but I couldn't find out how this method is implemented.

I assume that it's using busy-waiting by making a new thread (or a Task in .NET), but if so, I want to know how many second it Sleeps per loop (since Sleep(0) boosts my cpu fan).

Here's the code which I assume how it would look like:

HANDLE hOutputRead;
typedef void (*UserCallback)(size_t);
UserCallback OutputDataReceived;
void *Buffer;

bool exec(UserCallback callback, void *buffer /* string written to stdout */)
{
    OutputDataReceived = callback;
    Buffer = buffer;

    SECURITY_ATTRIBUTES sa;
    sa.nLength = sizeof(SECURITY_ATTRIBUTES);
    sa.lpSecurityDescriptor = NULL;
    sa.bInheritHandle = TRUE;

    HANDLE hOutputReadTemp, hOutputWrite;

    CreatePipe(&hOutputReadTemp, &hOutputWrite, &sa, 0);

    DuplicateHandle(
        GetCurrentProcess(), hOutputReadTemp,
        GetCurrentProcess(), hOutputRead,
        0, FALSE, DUPLICATE_SAME_ACCESS
    )   // to prevent hOutputReadTemp from being inherited to the launched process, 
        // we duplicate to hOutputRead with no inheritance priviledge, 
    CloseHandle(hOutputReadTemp);   // and close the original.

    STARTUPINFO si = {};
    si.cb = sizeof(STARTUPINFO);
    si.dwFlags = STARTF_USESTDHANDLES;
    si.hStdOutput = hOutputWrite;

    PROCESS_INFORMATION pi;
    if (!CreateProcess(NULL, "C:\\path\\to\\child.exe", NULL, NULL, TRUE, 0, NULL, NULL, &si, &pi))
        return false;

    CloseHandle(pi.hThread);
    CloseHandle(hOutputWrite);  // we won't be using this handle anymore

    CreateThread(NULL, 0, CheckPipeAsync, NULL, 0, NULL);
}

DWORD WINAPI CheckPipeAsync(LPVOID lpParameter) // if anything gets written to output, call OutputDataReceived()
{
    int size;

    while (PeekNamedPipe(hOutputRead, NULL, NULL, NULL, &size, NULL))  // busy wait
    {
        if (size == 0) // nothing written
        {
            Sleep(TIME_TO_SLEEP_PER_LOOP);
            continue;
        }

        DWORD bytesRead;
        ReadFile(hOutputRead, Buffer, size, &bytesRead, NULL);
        OutputDataReceived(size);   // callback when something was written to stdout
    }

    return TRUE;
}

reference (How to spawn console processes with redirected standard handles)

Please pardon my poor English.

Instead of sleep, it might be possible to create a worker thread and have that thread block while waiting for input. That's just an idea I haven't needed to do this before. — Dave S, Oct 29 '17 at 03:41
this is not asynchronous io. at first you need create your pipe end by call `CreateNamedPipe` with flag `FILE_FLAG_OVERLAPPED`. at second you not need create dedicated thread. you need bind pipe to *IOCP* (can use `BindIoCompletionCallback`) and do asynchronous read from it. for [example](https://stackoverflow.com/a/46613238/6401656) — RbMm, Oct 29 '17 at 08:45
A rough equivalent of `BeginRead` in C++ is `co_return`. Once coroutines are part of the language standard. If you are using MSVC, it already ships an implementation of the TS (see [`yield` keyword to become `co_yield` in VS 2017](https://blogs.msdn.microsoft.com/vcblog/2017/01/27/yield-keyword-to-become-co_yield-in-vs-2017/), for example). — IInspectable, Oct 29 '17 at 09:05
@IInspectable - equivalent of `BeginRead` is simply `ReadFile` on asynchronous handle. *callback* is set via `BindIoCompletionCallback` or `CreateThreadpoolIo`. and in place *state* is object inherited from `OVERLAPPED` — RbMm, Oct 29 '17 at 09:11
@RbMm: `BeginRead` isn't part of the C++ Programming Language. That call would go into the payload of an awaitable implementation used in conjunction with coroutines. The equivalent of C#'s `await` is (or will be) C++' `co_await`. Coroutines allow an asynchronous implementation without the need for manual multithreading. — IInspectable, Oct 29 '17 at 09:22
@IInspectable - nobody said that part. we both say about equivalent. and `ReadFile` on asyncronous handle which have attached iocp with callback - is exactly equivalent. and you can note that if use `BindIoCompletionCallback` also no any *manual multithreading.* - we not create any threads yourself direct. even not create iocp. simply build own callback to file handle. and all. — RbMm, Oct 29 '17 at 09:57
@RbMm: `Stream.BeginRead` returns an awaitable object in .NET. The *language-level* equivalent in C++ is an awaitable type as specified in the [Coroutine TS](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4680.pdf). Before responding, do consider the possibility, that you do not know .NET or C++ well enough to object in a meaningful fashion. — IInspectable, Oct 29 '17 at 10:31
@IInspectable - here said *I want to implement this in native C++ with Win32 API* - and awaitable object here based on `OVERLAPPED` or `IO_STATUS_BLOCK` — RbMm, Oct 29 '17 at 10:42
@Dave S, RbMm, IInspectable: Thank you, I should have known that ReadFile actually blocks the thread if there is nothing to read (I thought overlapped IO was only to wait for the client connection). Problem solved. I'm sorry for making such an absurd question. — vbstb, Oct 29 '17 at 10:46
@vbstb - `ReadFile` **not block** thread if you use asynchronous file handle — RbMm, Oct 29 '17 at 10:53
@vbstb - you can do very many things, based on your target and wish. if you want exactly asynchronous i/o (this is the most power) you can use i/o ports or apc completion (also can and events but this is the worst variant). for my look use iocp the best. however you not need create iocp yourself. and you not need create any threads yourself. you simply bind your callback to pipe via `BindIoCompletionCallback` and call `ReadFile`. when operation complete - your callback will called. this is almost analog of `BeginRead` — RbMm, Oct 29 '17 at 12:41
@RbMm - thanks, I'll go with that. I guess I really have to learn more about async stuff. — vbstb, Oct 29 '17 at 22:40
async io if understand and correct use it - very power and nice thing — RbMm, Oct 29 '17 at 22:42

Asynchronous output redirection in Win32

0 Answers0