Subprocess started from Java finishes using waitFor but streams aren't terminated

Question

I am using Java's ProcessBuilder to start a subprocess, which is another Java program that has to run in a separate JVM.

I start two Threads to read from the stdout and stderr streams from the Process, so that there is no hang if the stream buffers are full. The call to Process.waitFor returns but the streams aren't terminated.

The code I am using looks something like (command is a list of strings):

ProcessBuilder pb = new ProcessBuilder(command);

final Process p = pb.start();
final ByteArrayOutputStream outStream = new ByteArrayOutputStream();
final ByteArrayOutputStream errStream = new ByteArrayOutputStream();

Thread outputThread = new Thread() {
    @Override
    public void run() {
        try {
            IOUtils.copy(p.getInputStream(), outStream);
        } catch (IOException e) {
            e.printStackTrace();
        }
    };
};
outputThread.start();

Thread errorThread = new Thread() {
    @Override
    public void run() {
        try {
            IOUtils.copy(p.getErrorStream(), errStream);
        } catch (IOException e) {
            e.printStackTrace();
        }
    };
};
errorThread.start();

int returncode = p.waitFor();
outputThread.join();
errorThread.join();

If I run something else, such as "java -version" or "dir" or something, the code works fine. I have access to the Java code that I am trying to run, but I have never heard that you should call close() on System.out.

You may be interested in my question about closing Process' streams appropriate: http://stackoverflow.com/questions/7065067/how-to-close-std-streams-from-java-lang-process-appropriate Another useful link: http://mark.koli.ch/2011/01/leaky-pipes-remember-to-close-your-streams-when-using-javas-runtimegetruntimeexec.html — Fabian Barney, Sep 19 '11 at 15:00
If I use `IOUtils.closeQuietly` or even call `close()` on the streams before the `join()` calls, the program hangs on that call instead. — David Göransson, Sep 20 '11 at 10:09
This is most probably because `IOUtils.copy(in,out)` uses `InputStream.read(byte[])` and that is blocking and waiting for incoming data. Therefore the threads are running forever and therefore `join()` waits forever. You've to force closing here after a given timeout exceeds. Please try an implementation like the one I posted in my answer from the link I gave you. I'll bet it can handle this here. — Fabian Barney, Sep 20 '11 at 14:18

score 3 · Answer 1 · answered Sep 27 '11 at 04:01

3

Apache commons exec does all this for you. A lot more easier to do...

http://commons.apache.org/exec/

answered Sep 27 '11 at 04:01

sethu

8,181
7
39
65

score 0 · Answer 2 · edited May 23 '17 at 12:06

As I know from this website you must close all std-streams from Process object on your own. Regardless if used before or not. This seems highly related to G1 garbage collector (default since Java 7 afaik) which keeps pipes opened when not closed explicitly - even when the subprocess terminated.

I am not familiar with the internals here but the code I posted in my answer to my question works fine on a 24/7 system calling GhostScript and others a lot.

Other questions and answers on SO stating that you must close Process' std-streams explicit:

Process Builder waitFor() issue and Open file limitations

Subprocess started from Java finishes using waitFor but streams aren't terminated

2 Answers2