CreateProcess stdout with CreateNamedPipe Overlapped

Question

I'm trying to read the std output of an external process (pgdump ) that I started with CreateProcess. I got this working with anonymous pipes but then the output is blocked and no like when I execute it via the commandline( missing end output). I have read numerous posts and discovered that I need CreateNamedPipes with WaitForSingleObject but I can't seem to get it to work. This is my working code with anonymous pipe but blocked and I'm missing the end of the output

#include <QDebug>
#include <QString>

#include <windows.h>
#include <sstream>
#include <iostream>
#include <random>

int main()
{   
    #define BUFFERSIZE 256

    std::string program = "\"C:\\Program Files (x86)\\PostgreSQL\\10\\bin\\pg_dump.exe\"" +
            std::string( " --dbname=postgresql://postgresUser:PostGresql13@127.0.0.1:5432/employee -j1 -Fd -b -v -f "
                         "C:\\development\\myproject\\Build\\debug\\1-export.psql");

    HANDLE hReadStdOut = NULL;
    HANDLE hWriteStdOut = NULL;


    SECURITY_ATTRIBUTES saAttr;
    PROCESS_INFORMATION piProcInfo;
    STARTUPINFO siStartInfo;

    ZeroMemory( &saAttr, sizeof( SECURITY_ATTRIBUTES ));
    ZeroMemory( &piProcInfo, sizeof( PROCESS_INFORMATION ));
    ZeroMemory( &siStartInfo, sizeof( STARTUPINFO ));

    saAttr.nLength = sizeof(SECURITY_ATTRIBUTES);
    saAttr.bInheritHandle = TRUE;
    saAttr.lpSecurityDescriptor = NULL;

    OVERLAPPED sOverlap;


    if( !CreatePipe(&hReadStdOut, &hWriteStdOut, &saAttr, 0) )
    {
        std::ostringstream os;
        os << GetLastError();

        qDebug() << "create pipe error : " << QString::fromStdString( os.str());
    }


    TCHAR* szCmdline = new TCHAR[ program.size() + 1];
    szCmdline[ program.size()] = 0;
    std::copy( program.begin(), program.end(), szCmdline );

    siStartInfo.cb = sizeof(STARTUPINFO);
    siStartInfo.hStdError = hWriteStdOut;
    siStartInfo.hStdOutput = hWriteStdOut;
    siStartInfo.hStdInput = NULL;
    siStartInfo.dwFlags |= STARTF_USESTDHANDLES;

    BOOL bSuccess = CreateProcess( NULL, szCmdline, NULL, NULL, TRUE, CREATE_NO_WINDOW, NULL, NULL, &siStartInfo,&piProcInfo );

    if ( ! bSuccess )
    {
        std::ostringstream os;
        os << GetLastError();

        qDebug() << "create process error : " << QString::fromStdString( os.str());
    }
    else
    {
        CloseHandle( hWriteStdOut );

        DWORD err;
        DWORD nBytesRead;
        char buf[BUFFERSIZE + 1];

        int i(1);

        for(;;)
        {
            std::cout << "iteration " << std::to_string( i ) << std::endl;

            if( !ReadFile( hReadStdOut, buf, sizeof( buf), &nBytesRead, NULL) || !nBytesRead )
            {}

            if( GetLastError() == ERROR_SUCCESS )
            {
            }

            std::cout.flush();
            buf[nBytesRead] = '\0';
            std::string string_ = buf;
            std::cout << string_ << std::endl;

            std::size_t found = string_.find("contents of");

            if( !nBytesRead )
                break;

            i++;
        }

        CloseHandle(piProcInfo.hProcess);
        CloseHandle(piProcInfo.hThread);
    }

    return 0;
}

Answer 1

For example, if your pg_dump.exe just write one line to standard output. There are several ways:

1. std::cout << "Hello World!\n";
2. printf("Hello World!\n");
3. std::cout << "Hello World!\n"<< std::endl;
4. WriteFile(GetStdHandle(STD_OUTPUT_HANDLE), "Hello World!\n", 14, &dwWritten, NULL);

If the pg_dump.exe exit after write the line, the ReadFile will not blocked and return with the line "Hello World!".

However, if the pg_dump.exe doesn't exit after write the line and continue other work. The first two write ways (1., 2.) will result in ReadFile blocked. But if you use the third or fourth way the ReadFile will block and return with the line "Hello World!".

What's the difference here is the first two write ways (std::cout << "Hello World!\n"; and printf("Hello World!\n");) has no flush operation at end of write but the third or fourth way has. std::endl and WriteFile flush the output buffer.

Summary:

Flush the output to cause it to be written to the underlying stream (which may be a file, a terminal or a pipe). Standard output is flushed at the following conditions:

1. When the program ends normally.
2. Use std::endl at the end.
3. Use WriteFile.

You can check if it is your case.