4

I am working on an application that starts child processes using Boost's C++ Process library (http://www.highscore.de/boost/process0.5) and redirects the standard out of that process to a file using the below code: boost::process::initializers::bind_stdout(boost::iostreams::file_descriptor_sink goes here )

The above solution and code worked well.

However, now, I need to be able to print the child process's standard out to both a file and the console.

I have found the code below which seems that it should do the job:

#include <boost/process.hpp>
#include <boost/iostreams/device/file_descriptor.hpp>
#include <boost/iostreams/stream.hpp>
#include <iostream>

using namespace boost::process;
using namespace boost::process::initializers;
using namespace boost::iostreams;

int main()
{
    boost::process::pipe p = create_pipe();
    {
        file_descriptor_sink sink(p.sink, close_handle);
        execute(
            run_exe("/bin/echo"),
            set_cmd_line("echo hello"),
            bind_stdout(sink)
        );
    }
    file_descriptor_source source(p.source, close_handle);
    stream<file_descriptor_source> is(source);
    std::string s;
    while (std::getline(is, s))
    {
            std::cout << s << std::endl;
            //will write line by line to file here
    }
}

I have tried running the above code and indeed I was able to print the output to the console and a file. But there is one major issue with the above code (at least in my case), the while loop (while (std::getline(is, s))) only starts after the invoked process has exited, which means that all the output that was accumulated during the process run is printed at once once the process finishes.

Now, What I need is to be able to get the output from the invoked process during it's runtime and print it to both the console and the file as it comes (rather than only at the end).

(Since the processes that I start are out of my control and can take even a few good minutes to run, and currently the screen console is just empty until the process exits).

I have went over a lot of websites trying to find a solution for this one, as well as playing a lot with the code, but I can't seem to nail the solution yet.

How can this be accomplished? (Is it even possible with Boost.Process?) How would I be able to make that string/stream populate with the std out of the child process during it's runtime rather that only when it's finished?

MichaelEr
  • 75
  • 2
  • 8

1 Answers1

1

In principle it should work as you require.

Perhaps you need to flush. (Try using std::unitbuf).

Alternatively, you might be on Windows, which doesn't support async pipes readily. The docs describe a create_async_pipe function:

bp::pipe create_async_pipe()
{
#if defined(BOOST_WINDOWS_API)
    std::string name = "\\\\.\\pipe\\boost_process_async_io";
    HANDLE handle1 = ::CreateNamedPipeA(name.c_str(), PIPE_ACCESS_INBOUND |
        FILE_FLAG_OVERLAPPED, 0, 1, 8192, 8192, 0, NULL);
    HANDLE handle2 = ::CreateFileA(name.c_str(), GENERIC_WRITE, 0, NULL,
        OPEN_EXISTING, FILE_FLAG_OVERLAPPED, NULL);
    return make_pipe(handle1, handle2);
#elif defined(BOOST_POSIX_API)
    return create_pipe();
#endif
}

So, here's a version that combines it all, complete with tee device:

#include <boost/process.hpp>
#include <boost/iostreams/device/file_descriptor.hpp>
#include <boost/iostreams/stream.hpp>
#include <boost/iostreams/tee.hpp>
#include <boost/iostreams/filtering_stream.hpp>
#include <iostream>
#include <fstream>

namespace bp = boost::process;
namespace io = boost::iostreams;
using namespace bp;
using namespace bp::initializers;

bp::pipe create_async_pipe()
{
#if defined(BOOST_WINDOWS_API)
    std::string name = "\\\\.\\pipe\\boost_process_async_io";
    HANDLE handle1 = ::CreateNamedPipeA(name.c_str(), PIPE_ACCESS_INBOUND |
        FILE_FLAG_OVERLAPPED, 0, 1, 8192, 8192, 0, NULL);
    HANDLE handle2 = ::CreateFileA(name.c_str(), GENERIC_WRITE, 0, NULL,
        OPEN_EXISTING, FILE_FLAG_OVERLAPPED, NULL);
    return make_pipe(handle1, handle2);
#elif defined(BOOST_POSIX_API)
    return create_pipe();
#endif
}

int main()
{
    bp::pipe p = create_async_pipe();

    {
        io::file_descriptor_sink sink(p.sink, io::close_handle);
        execute(
            run_exe("/bin/sh"),
            set_cmd_line("sh -c \"echo started; sleep 3 && date\""),
            bind_stdout(sink)
        );
    }

    io::file_descriptor_source source(p.source, io::close_handle);
    io::stream<io::file_descriptor_source> is(source);

    std::cout << "Process is running, let's wait before printing\n";

    system("sleep 1");

    std::ofstream ofile("test.log");
    io::filtering_ostream filt(io::tee(std::cout << std::unitbuf, ofile << std::unitbuf));

    std::string s;
    while (std::getline(is, s))
        filt << s << "\n";
}

The commands have been chosen in such a way that you can verify that it works in async fashion.

If you want "true" asynchrony, you can also employ Boost Asio, see the documentation for details: http://www.highscore.de/boost/process0.5/boost_process/tutorial.html#boost_process.tutorial.asynchronous_i_o

sehe
  • 374,641
  • 47
  • 450
  • 633
  • Thanks a lot for that. After you have noted that it should work as-is, I have decided to double check the application I am executing as the child process, and it turns out that the problem was in the Python executable (Python buffers the standard out - here is a solution to this: http://stackoverflow.com/questions/107705/python-output-buffering) – MichaelEr Oct 18 '14 at 17:49
  • On another note, I was unable to compile the code above, I have got an error during compilation at this line: `io::filtering_ostream filt(io::tee(std::cout << std::unitbuf, ofile << std::unitbuf));` The exception was something along these lines: none of the 2 overloads could convert all the argument types. Any idea why does that happen and how to fix that? – MichaelEr Oct 18 '14 at 17:53
  • btw, I have tested my original code, and now it works just fine. Do you think that I should rather adjust to the code that you have posted anyway? – MichaelEr Oct 18 '14 at 17:54
  • @MichaelEr you can do what you prefer of course :) Thanks for reporting back – sehe Oct 18 '14 at 20:06