ProcessBuilder is holding a lock to the spawned process

Question

I am invoking an external JAVA process from my program. I am consuming the output generated by that spawned process like this:

DataInputStream dis = new DataInputStream(new BufferedInputStream(myProcess.getInputStream()));

From a Thread I am doing the following:

            while (dis.available() != 0) 
            {                   

                firstMesg = dis.readLine();


                if(firstMesg != null) 
                {
                    // processing with the message
                    //System.out.println(firstMesg);    
                }                                                                       
            } 


            try 
            {
                Thread.currentThread().sleep(SLEEP_TIME);
            }
            catch(Throwable e) 
            {
            }

I was giving SLEEP_TIME around 1 minute and everything was working just fine. Suddenly for a particular setup I have found Sys out (System.out.println) is taking awefully long time from the spawned process.

Can anyone point to me what happened? These two processes must be independent. However the invoker is holding reading from the invoked process. But the buffer should be large where the invoked process is writing. So there is no way it should get blocked.

I can see this in the ProcessBuilder Java doc:

The parent process uses these streams (#getInputStream(), #getErrorStream()) to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.

What does the external do? As here thread blocks on `dis.readLine();` you have to check what happened in external process. — Antoniossss, Oct 30 '13 at 11:08
I am doing this on a thread and continuously checking what the invoked process has produced. In between there is a sleep. — Exploring, Oct 30 '13 at 11:20
I had external process in mind - invoked one as you calling it. Anyway, either you have blocked external process by waiting too long in your main application, or process had some trouble generating output. All in all, you should't wait like that, but like Brian wrote, keep reading from process (this operation will block for you for as much as it needs to). Get some information about streams and stream operations. — Antoniossss, Oct 30 '13 at 11:21

score 1 · Answer 1 · answered Nov 20 '13 at 03:30

1

Take a look at the NuProcess library, it provides non-blocking (asynchronous) I/O for spawned processes. Disclaimer: I am the author of NuProcess.

answered Nov 20 '13 at 03:30

brettw

10,664
2
42
59

score 0 · Answer 2 · edited May 23 '17 at 11:50

0

You need to continually read, rather than read what's immediately available and then sleep. Note (also) that you should do the same for both stdout and stderr.

See this question for info on using a StreamGobbler to achieve this.

edited May 23 '17 at 11:50

Community

1
1

answered Oct 30 '13 at 11:08

Brian Agnew

268,207
37
334
440

If I carry on continually reading the output, in that case I shall get blocked. So I am reading whats available and then going to sleep and looping. – Exploring Oct 30 '13 at 11:23
Yes, you're blocked. But you're blocked on the thread you've spawned off. *Most of the time* your stdout and stderr consuming threads will be blocked waiting for your process to output something (unless your process is very verbose indeed) – Brian Agnew Oct 30 '13 at 11:43
I think you misunderstood the problem. The invoker process is not blocked. The invoked process is blocked. – Exploring Oct 30 '13 at 13:07

rolfl · Answer 3 · 2013-10-30T14:03:18.550

0

This is behaviour I would expect in your case. If a process produces stdout (or stderr for that matter) output it can't continue doing that forever without something actually doing something with that output. As your process produces output it will get buffered by the operating system until that buffer gets full. Your process reads from the buffer making more space again. If your program does not read then at some point the buffer will fill up, and the operating system will suspend the process until space comes available.

Create a thread in your Java process to read the STDOUT, and buffer it up in the Java side if you need to. On the other hand, you should probably process it as it arrives simply because it sounds like there is a lot of it.

EDIT: Based on comments... and other things....

DataInputStream.readLine() is deprecated, don't use it.

Instead of having a while loop that does:

while (dis.available() != 0) {...}

do

while ((line = br.readLine()) != null) { ....}

where br is a new BufferedReader(new InputStreamReader(myProcess.getInputStream()))

edited Oct 30 '13 at 14:03

answered Oct 30 '13 at 11:09

rolfl

17,539
7
42
76

I am already _doing this on a thread_ and continuously checking what the invoked process has produced. In between there is a sleep – Exploring Oct 30 '13 at 11:22
He is not continuously checking the proces.... he only checks it until the first moment when there's nothing available. `available()` will return `0` when there's nothing there, which will probably happen for the first fraction of a second at least while the process starts up.... as a consequence, nothing is read from the STDOUT. Consider using some other conditional in the while loop. – rolfl Oct 30 '13 at 11:34
Added some clarifications. – rolfl Oct 30 '13 at 14:03

ProcessBuilder is holding a lock to the spawned process

3 Answers3