11

Is there an easy way to read output from subprocess unbuffered? I'm trying to call a C program from Java, but apparently it's stdout block-buffered when connected to pipe and line-buffered only when connected to console. I cannot modify C program.

Maybe there is a way to fool the program into thinking it is connected to console? Bonus points for a solution that works on linux as well.

Norill Tempest
  • 143
  • 9
  • 1
    You could open an ssh to the same machine and run it that way. I suspect there is nothing simple to do this. – Peter Lawrey Sep 21 '12 at 16:00
  • "line buffering" occurs at the OS level. On Linux, for example, you'd do an "ioctl()" to set your terminal to "raw mode". Perhaps you can Google for "Java terminal I/O", "Java curses" or similar keywords? – paulsm4 Sep 21 '12 at 16:07
  • I already did a thorough search in the Internet and on SO. This seems to be a common problem, yet I haven't found a single solution for Windows – Norill Tempest Sep 21 '12 at 18:22
  • @paulsm4: on Windows, at least, the C runtime library typically does its own buffering, and this is probably the OPs issue. He's launching a C program as a subprocess, with the output piped to his Java program, but the output of the C program isn't being sent to the pipe in a timely manner because the runtime library is buffering it. On Linux you can use a pseudo-terminal, but to the best of my knowledge there is no good solution on Windows. – Harry Johnston Sep 23 '12 at 23:45
  • Just a thought: Did you try starting the C-Process indirectly via the cmd-shell? Something like this: `new ProcessBuilder("cmd", "/c", "", ...)`. – holgero Nov 29 '12 at 17:59
  • It doesn't change anything, probably cmd itself is also buffered... And it makes terminating application more difficult. – Norill Tempest Nov 29 '12 at 22:40
  • "I cannot modify C program..." Is it linked statically against the CRT, or can you drop in your own replacement for the CRT redistributable? In that case, you can probably just do something crude to replace a single function and forward all the others to the real CRT. – Nicholas Wilson Dec 19 '12 at 20:38

5 Answers5

1

It's not a great solution, but the runtime library probably doesn't buffer serial ports so if you're desperate enough you could use a null-modem emulator such as com0com or a derivative thereof.

Harry Johnston
  • 35,639
  • 6
  • 68
  • 158
0

Have you tried using environment variable BUF_1_=0?

Wu Yongzheng
  • 1,707
  • 17
  • 23
  • As far as I can tell from a Google search, this was at one point a proposed mechanism for suppressing buffering in the glibc library, but the developers rejected it. Do you have a better reference? (I've tried it with Visual Studio and it doesn't seem to work.) – Harry Johnston Oct 31 '12 at 20:01
-1

OK -

1) "Line buffered input" is a common idiom for console-mode programs, dating back to the original Unix and serial-mode VTxx terminals.

2) You can read "raw, unbuffered" I/O (a keystroke at a time, instead of a line at a time), but the details are OS specific. Whatever you need to do on a specific OS, you almost certainly can do from Java.

3) It sounds like you want to be able to intercept an "arrow up" key or "page down" key, as it happens, on a Windows keyboard. Perhaps for playing a game, or for interacting with a console-mode user interface.

4) There are several options. One you might wish to consider is the "robot" API, used for testing:

If that's not sufficient, please give more details about exactly how you're trying to get your Java program to interact with the C program (and clarify if the platform is indeed Windows and a DOS prompt).

paulsm4
  • 114,292
  • 17
  • 138
  • 190
  • I don't need raw input, line-buffered will suffice. I tried `Runtime.getRuntime().exec(command);` or `new ProcessBuilder(command).start();` to execute the C program, and then `process.getInputStream()` but it gave me a block-buffered stream. – Norill Tempest Sep 24 '12 at 00:50
-1

You could redirect the c output to a file and then tail the file from another thread in java using 

org.apache.commons.io.input.Tailer
Toon Krijthe
  • 52,876
  • 38
  • 145
  • 202
user1569047
  • 31
  • 2
  • Unfortunately, writing to a file will almost certainly also be buffered. – Harry Johnston Oct 13 '12 at 20:36
  • The user in the comment below says line buffering is fine. We want to redirect the output directly from the executable and not from java, maybe that works. – user1569047 Oct 14 '12 at 15:18
  • I've never heard of anything in Windows doing line buffering; typically, output is either block buffered or unbuffered. At any rate, if redirection to a pipe is block buffered, redirection to a file will almost certainly also be block buffered. This isn't a Java issue, it has to do with the behaviour of the C runtime library in the subprocess. – Harry Johnston Oct 14 '12 at 18:27
-1

The best way to do it is using Threads to read the output

What I would do is: using a Callable (a Runnable will also work) where I give the input from the process (process.getInputStream ()) and the output where I want to store the output. The same should be done for the StdErr. The resulting output can be read using whatever you like.

 public Object call () throws IOException {
        int bytesRead;
        byte[] b = new byte [ this.maxBlockSize ];

        try {
            while ( ( bytesRead = this.input.read ( b ) ) != -1 ) {
                this.output.write ( b, 0, bytesRead );
            }
        } finally {
            if ( this.output != null ) {
                this.output.close ();
            }
        }
        return null;
    }
azl
  • 109
  • 1
  • 8
  • That doesn't address the OPs problem. The output is being buffered in the child process, not in the parent process. – Harry Johnston Dec 03 '12 at 22:08
  • If it is done this way, it does not matter if it is blocking because it is read in a separate thread. – azl Dec 04 '12 at 14:07
  • 3
    You're missing the point, which is that he needs the output from the child process straight away, not at some indeterminate future time when the child's stdout buffer finally fills up. Making the read non-blocking means that his program can perhaps do some other job while it's waiting for the child process, but it doesn't help it to get the output any faster. (In particular, consider the case where the child process is waiting for input, but the parent process needs to see the output before it knows what input to send; the processes will deadlock.) – Harry Johnston Dec 04 '12 at 23:21