Boost process continuously read output

Question

I'm trying to read outputs/logs from different processes and display them in a GUI. The processes will be running for long time and produce huge output. I'm planning to stream the output from those processes and display them according to my needs. All the while allow my gui application to take user inputs and perform other actions.

What I've done here is, from main thread launch two threads for each process. One for launching the process and another for reading output from the process.

This is the solution I've come up thus far.

// Process Class
class MyProcess {
namespace bp = boost::process;
boost::asio::io_service mService; // member variable of the class
bp::ipstream mStream // member variable of the class
std::thread mProcessThread, mReaderThread // member variables of the class.

public void launch();
};

void
MyProcess::launch()
{
mReaderThread = std::thread([&](){
std::string line;
while(getline(mStream, line)) {
std::cout << line << std::endl;
}
});

mProcessThread = std::thread([&]() {
auto c = boost::child ("/path/of/executable", bp::std_out > mStream, mService);

mService.run();
mStream.pipe().close();
}
}


// Main Gui class
class MyGui
{
MyProcess process;
void launchProcess();
}

MyGui::launchProcess()
{
process.launch();
doSomethingElse();
}

The program is working as expected so far. But I'm not sure if this is the correct solution. Please let me know if there's any alternative/better/correct solution

Thanks, Surya

Answer 1

The most striking conceptual issues I see are

1. Process are asynchronous, no need to add a thread to run them.¹

2. You prematurely close the pipe:

   mService.run();
   mStream.pipe().close();
   

   Run is not "blocking" in the sense that it will not wait for the child to exit. You could use wait to achieve that. Other than that, you can just remove the close() call.

   With the close means you will lose all or part of the output. You might not see any of the output if the child process takes a while before it outputs the first data.

3. You are accessing the mStream from multiple threads without synchronization. This invokes Undefined Behaviour because it opens a Data Race.

   In this case you can remove the immediate problem by removing the mStream.close() call mentioned before, but you must take care to start the reader-thread only after the child has been initialized.

   Strictly speaking the same caution should be taken for std::cout.

4. You are passing the io_service reference, but it's not being used. Just dropping it seems like a good idea.

5. The destructor of MyProcess needs to detach or join the threads. To prevent Zombies, it needs to detach or reap the child pid too.

   In combination with the lifetime of mStream detaching the reader thread is not really an option, as mStream is being used from the thread.

Let's put out the first fixes first, and after that I'll suggest show some more simplifications that make sense in the scope of your sample.

First Fixes

I used a simple bash command to emulate a command generating 1000 lines of ping:

Live On Coliru

#include <boost/process.hpp>
#include <thread>
#include <iostream>
namespace bp = boost::process;

/////////////////////////
class MyProcess {
    bp::ipstream mStream;
    bp::child mChild;
    std::thread mReaderThread;

  public:
    ~MyProcess();
    void launch();
};

void MyProcess::launch() {
    mChild = bp::child("/bin/bash", std::vector<std::string> {"-c", "yes ping | head -n 1000" }, bp::std_out > mStream);

    mReaderThread = std::thread([&]() {
        std::string line;
        while (getline(mStream, line)) {
            std::cout << line << std::endl;
        }
    });
}

MyProcess::~MyProcess() {
    if (mReaderThread.joinable()) mReaderThread.join();
    if (mChild.running()) mChild.wait();
}

/////////////////////////
class MyGui {
    MyProcess _process;
  public:
    void launchProcess();
};

void MyGui::launchProcess() {
    _process.launch();
    // doSomethingElse();
}

int main() {
    MyGui gui;
    gui.launchProcess();
}

Simplify!

In the current model, the thread doesn't pull it's weight.

I you'd use io_service with asynchronous IO instead, you could even do away with the whole thread to begin with, by polling the service from inside your GUI event loop².

If you're gonna have it, and since child processes naturally execute asynchronously³ you could simply do:

Live On Coliru

#include <boost/process.hpp>
#include <thread>
#include <iostream>

std::thread launch(std::string const& command, std::vector<std::string> args = {}) {
    namespace bp = boost::process;

    return std::thread([=] {
        bp::ipstream stream;
        bp::child c(command, args, bp::std_out > stream);

        std::string line;
        while (getline(stream, line)) {
            // TODO likely post to some kind of queue for processing
            std::cout << line << std::endl;
        }

        c.wait(); // reap PID
    });
}

The demo displays exactly the same output as earlier.

---

¹ In fact, adding threads is asking for trouble with fork

² or perhaps idle tick or similar idea. Qt has a ready-made integration (How to integrate Boost.Asio main loop in GUI framework like Qt4 or GTK)

³ on all platforms supported by Boost Process