Boost.Process: captured stdout is buffered until size X

Question

I am using boost.process to start another process. I want to capture the stdout and print it myself. The problem is, that the output is only printed in chunks or when stopping the subprocess. The test subprocess is a python script which calls echo "test" 20 times per second.

void ExternalAppLauncher::setup()
{
    boost::process::pipe stdout_p = boost::process::create_pipe();
    boost::process::pipe stderr_p = boost::process::create_pipe();

    {
        file_descriptor_sink stdout_sink(stdout_p.sink, close_handle);
        file_descriptor_sink stderr_sink(stderr_p.sink, close_handle);
        file_descriptor_source stdout_source(stdout_p.source,  close_handle);
        file_descriptor_source stderr_source(stderr_p.source,  close_handle);

        out_stream.open(stdout_source);
        err_stream.open(stderr_source);

        childProcess.reset(new child(execute(
                                         set_args(args),
                                         bind_stdout(stdout_sink),
                                         bind_stderr(stderr_sink),
                                         inherit_env(),
                                         // Guarantees that the child process gets killed, even when this process recieves SIGKILL(9) instead of SIGINT(2)
                                         on_exec_setup([](executor&)
        {
            ::prctl(PR_SET_PDEATHSIG, SIGKILL);
        })
                                     )));
    }
}

// runs in another thread
void ExternalAppLauncher::run()
{
    std::string s;
    while (std::getline(err_stream, s))
    {
        std::cout << s;
    }
}

This prints the output only every 10 seconds, probably because the buffer needs to be full before it is forwared? When I dont call bind_stdout() the output appears immediately on the console. What could solve this problem?

Thanks!

score 1 · Answer 1 · edited May 23 '17 at 10:30

As I found out in the thread How to capture standard out and print to both the console and a file during process runtime (C++/Boost.Process) the python script I ran was the culprit. Deactivating the buffering for the python script with export PYTHONUNBUFFERED=1 solved it. The env var can be passed in to the application with set_env:

    childProcess.reset(new child(execute(
                                     set_args(args),
                                     start_in_dir(workingDir),
                                     bind_stdout(stdout_sink),
                                     bind_stderr(stderr_sink),
                                     inherit_env(),
                                     set_env(std::vector<std::string> {"PYTHONUNBUFFERED=1"}))));

Boost.Process: captured stdout is buffered until size X

1 Answers1