2

I'm writing a terminal wrapper for a command-line program in Java, and I spawn the subprocess using ProcessBuilder. To send keystrokes to the subprocess, I just write e.getKeyChar() from the GUI straight to the OutputStream given by proc.getOutputStream(). To receive output from the subprocess, I basically have a while loop that reads from the subprocess's stdout:

while ((b = br.read()) != -1) {
    System.out.println("Read "+b);
    bb[0] = (byte) b;
    // call an event listener with the read byte
    listener.dataReceived(bb);
}

This works, only if I immediately flush the output on both ends. That is, I have to flush every user input and the subprocess has to flush its own stdout in order for stuff to happen. Otherwise, read() blocks, waiting for data, which is never actually sent (subprocess' stdout just keeps buffering). How can I get I/O going?

Example terminal subprocess:

#include <stdio.h>

int main() {
    char c;
    while((c = getchar()) != -1) {
        printf("Got: %d\n", c);
        // doesn't work in my Java program if the next line isn't present
        fflush(stdout);
    }
    return 0;
}

I'm running on Ubuntu 10.10 with Sun Java 6.

erjiang
  • 44,417
  • 10
  • 64
  • 100

3 Answers3

5

You cannot read data from a file until the data has been written to the disk. You cannot read data from a socket or pipe until the data has been put in the pipe/socket's buffer.

Your java program has no control(*) over when an external process flushes its output and writes data to the disk/pipe buffer/socket buffer. You are totally at the mercy of the buffering behavior of the external program. This is true on every operating system and in every programming language.

Every network programmer has to deal with this, so just deal with it.

(*) - Occasionally some programs (like cat for one) have options (-u) to instruct the program to use unbuffered output. Otherwise you are mercy of the

socket puppet
  • 3,191
  • 20
  • 16
1

You aren't running your I/O read loop from the event dispatch thread are you?

You should run the I/O read from the sub-process in a separate thread (if you are not already doing this). Immediate flush to the subprocess per GUI key-press is probably best; unless you want to support some kind of 'read a whole line at a time' thing.

Justin
  • 4,437
  • 6
  • 32
  • 52
  • 1
    This is what I'm currently doing. I have a separate thread reading from the subprocess (since `read()` blocks) and it notifies an event handler in the main thread. Keystrokes are immediately sent to the subprocess and flushed. – erjiang Nov 21 '10 at 00:44
1

Many many runtime libraries (e.g., I know that libc does this, and wouldn't be at all surprised if others do too) will buffer their output by default except when the output is to a terminal. This enormously increases the efficiency of data handling when dealing with many lines (e.g., in a normal pipeline) but when there is only a small amount of information, it hurts a lot. If you have access to the source of the subprocess, it's definitely best to update the code by turning off buffering or adding flushes.

But that's not always possible, especially when dealing with third-party code. The best other fix I know of in that case is to use a tool like Expect to trick the subprocess. Internally, Expect knows how to pretend to be a terminal (using ptys on Unix and godawful hacks on Windows) so tricking the other programs into turning off (or at least reducing) their buffering. There is a script – unbuffer – for Expect that makes it focus specifically on this sort of use. (In general it can do a lot more than just dealing with unruly buffering, but it's the best fix anyway.)

Donal Fellows
  • 133,037
  • 18
  • 149
  • 215