Java exec - output of interactive process is held till process has terminated

Question

I have a problem with an interactive process in Java. I have threads to read STDOUT and STDERR and a thread to handle input for the process. But there are no data available within the STDOUT stream until the process has terminated. Then is the entire output printed at once.

DBG    | Pipe action-STDERR started
DBG    | Pipe action-STDIN started
DBG    | Pipe action-STDOUT started

STDIN  | Try to put some input.
STDIN  | I cannot see any output.
STDIN  | Nevertheless the interaction works.
STDIN  | It works on background.
STDIN  | Let's terminate the process to see the truth.
STDIN  | quit

STDOUT | Enter some text, please: The text is 'Try to put some input.'
STDOUT | Enter some text, please: The text is 'I cannot see any output.'
STDOUT | Enter some text, please: The text is 'Nevertheless the'
STDOUT | Enter some text, please: The text is 'interaction works.'
STDOUT | Enter some text, please: The text is 'It works on background.'
STDOUT | Enter some text, please: The text is 'Let's terminate the process to see the truth.'
STDOUT | Enter some text, please: The text is 'quit'
STDOUT | Bye!
DBG    | Trying to kill thread action-STDOUT
DBG    | Trying to kill thread action-STDERR
DBG    | Trying to kill thread action-STDIN
DBG    | Pipe action-STDERR finished
DBG    | Finished
DBG    | Pipe action-STDIN finished
DBG    | Pipe action-STDOUT finished

The lines prefixed with STDOUT are lines written by the process. The lines prefixed with STDIN are lines written by me. Lines prefixed by DBG are lines written by the tested Java program as debug information. Let's try to execute the same process in system console.

Enter some text, please: Text
The text is 'Text'
Enter some text, please: quit
The text is 'quit'
Bye!

The behavior fullfits my expectation. I'm asked for some input. I do it and get the answer.

I'm surprissed, that I've found several posts on web including Stackoverflow, but without any answer marked as acceptable solution. (For example Problem reading InputStream from Java Process.) It seems Java developers haven't ever been dealing with execution of an interactive process. Strange is, that output of non-interactive process (as ping) is appearing sequentially, while the process is running. Without any problem. But when the process is waiting for user input, the output is somehow blocked.

Answer 1

Regrettably it is not a bug, it is a feature. Either on Linux or on Windows pipes are implemented like this. They are buffered in order to increase performace. One process produces data and the second one takes them over. There is no interaction needed and this model works in 99% cases as well.

However the console is another kind of application. The system console is a part of OS and it works a bit other way the our console written in Java. The system console of course prints all data immediately. All works fine. Then the same process executed from our code does not print prompts, albeit it shall work. What's wrong? There is no buffer between system console and process. But there is a buffer between our process and our Java code on the contrary. And that's the cause.

The process does not flush output automatically. Regardless to the language the process is written in. Process written in Java, Perl or C never prints prompts until explicit flush has been added to its code. If you can affect the code of a process you will interact with, update the code (pseudocode follows):

print 'Give me a number '
flush STDOUT
number = get STDIN
print 'You wrote ' + number
flush STDOUT

There is not another solution. If the third-party application does not flush its outputs, you cannot use it in your console. You can hardly evade the buffering. Nevertheless it is possible somehow. There are libraries or you can find an open source application which can interact with problematic processes and inspect its code.

Tip: Check out Apache Commons Exec. It is a library specialized in process executing.