2

I want to read the stdout of a process right as it is generated. The process will send information for a progress indicator, so it doesn't make sense that I get the information all at once, which I do and which is the problem. I tried to use Scanner class as suggested in a post, but I still get the output only after the process has finished. I realize this question has been asked before, but it hasn't been answered.

You will probably want to look at class StreamGobblerOutput first.

    public List<String> executeCall(String fileName) 
    {
    StringBuilder sbOutput = new StringBuilder();
    StringBuilder sbError = new StringBuilder();

    File file = new File(fileName);
    try ( BufferedReader br = new BufferedReader(new FileReader(file)) ) {
        String line;
        while ((line = br.readLine()) != null) {
           String [] parts = line.split("\\s");
           if(parts.length<2) {
               sbError.append("Command too short for call: " + parts[0]);
               continue;
           }

            List<String> args = new ArrayList<String>();
            args.add ("sfb.exe");
            for(int i = 1; i <parts.length; ++i) {
                args.add (parts[i]);
            }
            args.add (sfbPassword);

            ProcessBuilder pb = new ProcessBuilder (args);
            pb.directory(new File(Support.getJustThePathFromFile(file)));
            Map<String, String> envs = pb.environment();
            String path = envs.get("Path");
            envs.put("Path", Paths.get(".").toAbsolutePath().normalize().toString() + ";" +path);


            //pb.redirectOutput(new Redirect() {});
            Process p = pb.start();

            String outputPathPrefix = pb.directory().getCanonicalPath();

            // any output?
            StreamGobblerOutput outputGobbler = new StreamGobblerOutput(p.getInputStream(), outputPathPrefix);
            outputGobbler.start();

            // any errors?
            StreamGobblerError errorGobbler = new StreamGobblerError(p.getErrorStream());
            errorGobbler.start();


            try
            {
                p.waitFor();
            }
            catch (InterruptedException e)
            {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }

            sbOutput = outputGobbler.getOutput();
            sbError = errorGobbler.getErrors();

            String rootPath= Support.getJustThePathFromFile(new File(fileName));
            File rootFile = new File(rootPath + "/..");
            String rootFolder = rootFile.getCanonicalFile().getName();
            System.err.println("rootFolder: " + rootFolder);
            mainApp.addModifiedFiles(outputGobbler.getModifiedFileNames(), rootFolder);

        }
    } catch ( IOException ex) {
        sbError.append(ex.getMessage());
    }

    mainApp.addOutput(sbOutput.toString());
    mainApp.addError(sbError.toString());

    return;

}

    private class StreamGobblerOutput extends Thread {
        private InputStream is;
        private String outputPathPrefix;
        private StringBuilder sbOutput;
        private List<String> modifiedFileNames;
        private Scanner scanner;

        private StreamGobblerOutput(InputStream is, String outputPathPrefix) {
            this.is = is;
            this.outputPathPrefix = outputPathPrefix;
            sbOutput = new StringBuilder();
            modifiedFileNames = new ArrayList<String>();
            scanner = new Scanner(is);
        }

        public StringBuilder getOutput() {
            return sbOutput;    
        }

        public List<String> getModifiedFileNames() {
            return modifiedFileNames;       
        }

        @Override
        public void run() {
            //create pattern
            Pattern patternProgress = Pattern.compile("\\((\\d+)%\\)");

            //InputStreamReader isr = new InputStreamReader(is);
            //BufferedReader br = new BufferedReader(isr);
            String ligne = null;
            while (scanner.hasNextLine()) {
                ligne = scanner.nextLine();

                sbOutput.append(ligne);
                sbOutput.append("\r\n");
                //bw.write("\r\n");

                Matcher mProgress = patternProgress.matcher(ligne);
                if (mProgress.find()) {
                    int percentage = Integer.parseInt(mProgress.group(1));
                    System.err.println("percentage=" + percentage);
                    mainApp.mainWindowController.setProgressExecute(percentage/100.0);
                }
            }
            mainApp.mainWindowController.setProgressExecute(1.0);
            if (scanner != null) {
                scanner.close();
             }
        }
    }

    private class StreamGobblerError extends Thread {
        private InputStream is;
        private StringBuilder sbError;
        private Scanner scanner;

        private StreamGobblerError(InputStream is) {
            this.is = is;
            sbError = new StringBuilder();
            scanner = new Scanner(is);
        }

        public StringBuilder getErrors() {
            return sbError;     
        }

        @Override
        public void run() {
            //InputStreamReader isr = new InputStreamReader(is);
            //BufferedReader br = new BufferedReader(isr);
            String ligne = null;
            while (scanner.hasNextLine()) {
                ligne = scanner.nextLine();
                sbError.append(ligne);
                sbError.append("\r\n");
            }
            if (scanner != null) {
                scanner.close();
             }
        }
    }

Update: I tried redirecting the output to a file and reading from it, but it appears this runs into the same buffering problem as the previous implementation: I get only two data points.

As a workaround, I will have to ask the creator of the .exe to include 4100 extra characters in each line showing the progress.

Adder
  • 5,708
  • 1
  • 28
  • 56
  • Have you got access to the code of the process which generates output? Perhaps you can do flushes there on some interval of time/after n number of writes? – P.An Jul 13 '17 at 09:00
  • I do not have access to the code of the process i call, but when i call it from the console, it seems to output the progress indicator data one by one i.e.: it seems to be flushed after each data point. – Adder Jul 13 '17 at 09:08
  • @Adder - is the external program your are invoking C/C++ based ? – zeppelin Jul 17 '17 at 14:50
  • you can output the exe to a file then tail the file to check whether output the progress indicator one by one? – andy Jul 18 '17 at 08:42
  • I tried redirecting it to file, but when i read the file, I have the buffering problem again. – Adder Jul 19 '17 at 10:22

2 Answers2

2

If your external process is C/C++ (stdio) based, than this is most likely a block buffering issue:

stdio-based programs as a rule are line buffered if they are running interactively in a terminal and block buffered when their stdout is redirected to a pipe. In the latter case, you won't see new lines until the buffer overflows or flushed.

see this answer for more details, and some possible workarounds.

Please also note that according to this, line buffering is not an option on Win32:

_IOLBF For some systems, this provides line buffering. However, for Win32, the behavior is the same as _IOFBF - Full Buffering.

so if you choose to modify the "exe" program to set a proper output mode with setvbuf, you would have to use:

_IONBF No buffer

instead.

zeppelin
  • 8,947
  • 2
  • 24
  • 30
  • Thanks for your reply. How would the C program need to be modified? – Adder Jul 19 '17 at 09:23
  • https://www.gnu.org/software/libc/manual/html_node/Controlling-Buffering.html This way, i guess. – Adder Jul 19 '17 at 10:20
  • @Adder, yep, of see this links if this is a windows binary: [setvbuf](https://msdn.microsoft.com/en-us/library/86cebhfs.aspx), but take a note that line buffering is not supported under win32. – zeppelin Jul 19 '17 at 10:56
0

From the Javadocs

Optionally, a PrintStream can be created so as to flush automatically; this means that the flush method is automatically invoked after a byte array is written, one of the println methods is invoked, or a newline character or byte ('\n') is written.

Reference: http://docs.oracle.com/javase/8/docs/api/java/io/PrintStream.html

  1. One way to do this is to have output stream flush() after every write. System.out.flush()

  2. You could also define your own flushable PrintStream and use that.

worker_bee
  • 451
  • 4
  • 11
  • My process generating the data points is not written in java and is only indirectly accessible to me.. – Adder Jul 20 '17 at 07:38