Pipe between java processes on command shell not reliable working

Question

I'm trying to pipe text between two java programs. For the sake of simplicity I'm presenting this code:

import java.io.DataInputStream;
import java.io.IOException;

public class Test {
    public static void main(String[] args) throws IOException {
        DataInputStream stdin = new DataInputStream(System.in);
        String completeText = ""; 

        while (stdin.available() > 0) {
            byte[] tempByte = { stdin.readByte() };
            completeText += new String(tempByte);
        }

        System.out.println(completeText);
    }
}

When doing the following on linux or windows, the text seems to get omitted as if the pipe was blocking or lost quite random. Sometimes everything gets through, sometimes not:

echo "omg wtf" | java Test | java Test

Any ideas on this? The slower the cpu the more often the text gets through, it seems. Is "available" returning the wrong result for any reason when the input is piped from java System.out.println()?

Cheers!

Itay Maman · Answer 1 · 2011-04-25T21:15:16.557

First, The available() method is not a reliable way to determine whether a stream is exhausted. A reliable indication to an end-of-stream is by checking the return value of the read() method (< 0 means end-of-stream).

In short, available() can return false (which will terminate the loop), if the stream is momentarily empty. If the pipe is still active, this situation will change as soon as the process on the other end of the pipe writes some bytes into it. To be sure that all data was read you need to check for end-of-stream.

Second, if you want to read characters (and concatenate them into a string) you should read characters from a Reader (and not bytes from a stream). This will allow your code to handle unicode characters.

Third, concatenation of large chunks of characters is will be faster if you use a StringBuilder (rather than a plain String).

Finally, if you only need to read bytes, you can use the input-stream directly (no need to wrap it with a DataInputStream).

Here's how I'd write it:

Reader r = new InputStreamReader(System.in);
StringBuilder sb = new StringBuilder();
while(true) {
  int ch = r.read();
  if(ch < 0)
    break;
  sb.append((char) ch);
}

System.out.println(sb.toString());

score 3 · Answer 2 · answered Apr 25 '11 at 20:59

available() is not reliable for piped input. It checks if there is data in the current process' input buffer. It has no way to check if preceding (pipe-wise) process is about to send some data.

In your case blocking read is an acceptable solution:

public class Test {
    public static void main(String[] args) throws IOException {
        DataInputStream stdin = new DataInputStream(System.in);
        StringBuilder completeText = new StringBuilder(); 
        byte[] tempByte = new byte[1024];
        int len = 0;  
        while ((len = stdin.read(tempByte)) != -1) {
            completeText.append(new String(tempByte, 0, len));
        }
        System.out.println(completeText.toString());
    }
}

I also added StringBuilder as this is the "correct" java way to concatenate strings.

Pipe between java processes on command shell not reliable working

2 Answers2

Linked