3

In my Java application, I need to execute some scripts as subprocesses and monitor the output on stdout from Java so that I can react when necessary to some output.

I am using apache commons-exec to spawn the subprocess and redirect stdout of the executed script to an input stream.

The problem that I am having is that when reading from the stream, the Java process is blocked until the subprocess is finished execution. I cannot wait until the end of the subprocess to react to the output, but I need to read it asynchronously as it becomes available.

Below is my Java code:

public class SubProcessReact {
    public static class LogOutputStreamImpl extends LogOutputStream {

        @Override
        protected void processLine(String line, int logLevel) {
            System.out.println("R: " + line);
        }
    }

    public static void main (String[] args) throws IOException, InterruptedException {
        CommandLine cl = CommandLine.parse("python printNumbers.py");
        DefaultExecutor e = new DefaultExecutor();
        ExecuteStreamHandler sh = new PumpStreamHandler(new LogOutputStreamImpl());
        e.setStreamHandler(sh);

        Thread th = new Thread(() -> {
            try {
                e.execute(cl);
            } catch (IOException e1) {
                e1.printStackTrace();
            }
        });

        th.start();
    }
}

For this example, the subprocess is a python script which counts upwards with a one second delay between outputs so that I can verify that the Java code is responding as data comes in.

Python Code:

import time

for x in range(0,10):
    print x
    time.sleep(1)

I would expect LogOutputStreamImpl to print each line as it comes, but what is actually happening is that it reading the stream blocks until the subprocess is completed, and then all of the output is printed.

Is there something I could do to make this work as I intend?

apatrick
  • 194
  • 2
  • 14

1 Answers1

3

Why use a third-party library to do something Java SE already does well? Personally, I prefer to depend on as few external libraries as possible, in order to make my programs easily portable and to reduce the points of failure:

ProcessBuilder builder = new ProcessBuilder("python", "printNumbers.py");
builder.inheritIO().redirectOutput(ProcessBuilder.Redirect.PIPE);
Process process = builder.start();

try (BufferedReader reader = new BufferedReader(
    new InputStreamReader(process.getInputStream()))) {

    reader.lines().forEach(line -> System.out.println("R: " + line));
}

process.waitFor();
VGR
  • 40,506
  • 4
  • 48
  • 63
  • Thank you for your answer, but this does not solve my problem. I need to be able to read each line as it comes in to the stream. With this solution, I have the same issue that is present in my original code, I can only get the entire result after the script finishes execution. I need to be able to read each line as it comes in. – apatrick Dec 20 '15 at 17:16
  • 3
    @apatrick: In addition, to reading one line at a time in your java code, you might need `-u` flag, to disable buffering of `python`'s stdout/stderr stream. Otherwise, the output is block-buffered when redirected. See [my answer to a similar question but for `Python`](http://stackoverflow.com/a/17698359/4279) – jfs Dec 20 '15 at 20:38
  • @J.F.Sebastian This solved my problem. I did not think that the problem would be on the Python side. Thanks you very much! – apatrick Dec 20 '15 at 21:07
  • 2
    @apatrick: you're welcome. Btw, making the streams fully buffered if they are redirected to a pipe/file is common for C stdio-based programs, it is not Python specific e.g., that is why `grep` has `--line-buffered` option. The rational is that If there is no user to look at the output then the block-buffering mode can improve performance. See [related question](http://stackoverflow.com/q/20503671/4279). – jfs Dec 20 '15 at 21:16
  • also, the return type if `reader.lines()` is a stream. If you want to only read it for a while, instead of calling .forEach(), turn it into an iterable so you can easily `break` out whenever you see something important. – Ajax Jun 15 '23 at 00:09