How to read from InputStream that is constantly updating

Question

I am developing an applicaton in Java that basically just collects input from user via GUI and feeds this data to a python script that does some file system operations. This python script is executed via Java Process class. Each script simply prints some data to its stdout, which I parse in the main java application when I call process.getInputStream().

I have one python script that basically does tail -f on a log file and prints it line by line. My java program parses this output using InputStream from that particular process. However, I feel like it's incredibly slow. I am appending this log to a JavaFX TextArea and it usually takes around 20s after the python script is executed that my java program starts picking up anything, but when I run the python script by itself, it runs as expected.

What I have tried so far: Parsing it using buffered reader like this:

InputStream is = this.currentTask.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(is, StandardCharsets.UTF_8));
String line;
while (this.currentTask.isAlive()){
   line = reader.readLine();
   if (line != null){
           final String finalLine = line;
           // JavaFX stuff so that the appending is on the main thread
           Platform.runLater(() -> output.appendText(finalLine + "\n"));
           continue;
   }
   // wait for the log to be updated 
   Thread.sleep(100);
}

When I step through the lines with a debugger, it gets stuck for a very long time in th line = reader.readLine(); part. Switching the reading method for another one provided by the BufferedReader(like read with a specified char buffer) does not help at all.

Any idea what might be the problem?

EDIT: added MCVE

Java program:

public static void main(String[] args) {
    Process process = null;
    try {
        process = Runtime.getRuntime().exec("python" + " " + "main.py");
    } catch (IOException e) {
        e.printStackTrace();
    }
    try {
        InputStream is = process.getInputStream();
        BufferedReader reader = new BufferedReader(new InputStreamReader(is, StandardCharsets.UTF_8));
        String line;
        while (process.isAlive()){
            line = reader.readLine();
            if (line != null){
                System.out.println(line);
                continue;
            }
            // wait for the log to be updated
            Thread.sleep(100);

        }
    } catch (IOException | InterruptedException e) {
        e.printStackTrace();
    }
}

Python program:

import time

i = 0
while True:
    print("Output ", i)
    i += 1
    time.sleep(0.5)

I noticed that the whole thing works as it should when I remove the sleep from python script. However, in the real app, the sleep is needed.

Maybe if you read it char by char instead of line by line you get a better user experience. — jpuriol, Jul 08 '20 at 09:29
`reader.readline()` will block until a complete line of text (i.e. text terminated by a newline or by the end of the stream) is available. So it's likely that your python script is producing output but not sending a newline, or perhaps not flushing the entire output. (Note also that your `Thread.sleep(...)` is redundant due to `readline()` blocking.) As suggested, create and post a [mre] that others can use to reproduce the issue. (One simple JavaFX application and the simplest python script that reproduces the issue should suffice.) — James_D, Jul 08 '20 at 12:28
_Any idea what might be the problem?_ I'm not a friend of guessing, please provide a MCVE or [How to create a Minimal, Reproducible Example](https://stackoverflow.com/help/minimal-reproducible-example) as pointed out by @kleopatra. Also note that having a MCVE allows others to answer your question effectively. — Ivo Mori, Jul 08 '20 at 12:29
Don't forget to accept your own answer to the question so that this question doesn't show up as an unanswered question anymore. — Ivo Mori, Jul 12 '20 at 04:13
this most likely a problem of observer pattern i believe correct me if i misunderstood — silentsudo, Dec 28 '20 at 05:32

score 0 · Accepted Answer · answered Jul 08 '20 at 18:22

Ok I just solved it. Credit to this SO question.

Java wasn't the problem, python simply wasn't flushing its output often enough and Java had nothing to read, that's all. I have to manualy flush my python print() functions and everything works as it should.

How to read from InputStream that is constantly updating

1 Answers1