Asynchronous capture from a process output not working properly

Question

I'm developing a control for a website, where user can upload his PDF, and after the upload a 3rd party CLI-tool launches to verify PDF against certain profile and generate a corresponding report. The tool mentioned is callas pdfToolbox 4 (available here http://www.callassoftware.com/callas/doku.php/en:download[^])

The problem is that on my control at the website I need to display in real-time progress bar of checking the PDF-file. All the AJAX-stuff for this functionality is already written (ajax-postbacks, updates of progress-bar, etc), but there's a problem with asynchronous updates from the process, which launches the pdf-checking tool.

If you launch the tool from command-line window, you can see that, it generates output into standard output stream, which contains progress updates (in percents), as well as possible messages about errors in the PDF-file.

However, if the tool is launched by the process which I create in my web-control, I don't receive the OutputDataReceived events until the check has been finished, and then many OutputDataReceived events come at once, one after one.

My code is the following (I've written a small console-app to test things faster):

class Program
{

        static string appString = "path-to-callas-cli";
        static string argString = "path-to-pdf-and-path-to-report-and-path-to-callas-profile";

         static void Main(string[] args)
        {
             ProcessStartInfo pInfo = new ProcessStartInfo(appString, argString);
            pInfo.UseShellExecute = false;
            pInfo.CreateNoWindow = true;
            pInfo.RedirectStandardOutput = true;
            pInfo.RedirectStandardError = true;
            pInfo.RedirectStandardInput = true;
            pInfo.ErrorDialog = true;

            Process process = new Process();
            process.StartInfo = pInfo;
            process.OutputDataReceived += new DataReceivedEventHandler(process_OutputDataReceived);
            process.Exited += new EventHandler(process_Exited);
            process.ErrorDataReceived += new DataReceivedEventHandler(process_ErrorDataReceived);
            process.Start();

            process.BeginOutputReadLine();
            process.BeginErrorReadLine();

            process.WaitForExit();

            Console.ReadKey();
        }

        static void process_ErrorDataReceived(object sender, DataReceivedEventArgs e)
        {

        }

        static void process_Exited(object sender, EventArgs e)
        {
        }

        static void process_OutputDataReceived(object sender, DataReceivedEventArgs e)
        {
            Console.WriteLine("Received async output: " + e.Data);
        }
}

As I've said, all the output from Callas CLI comes at once in the end (while the check takes about 35 seconds). To test my own code, I've created a small console app, which outputs numbers 10-20-30-....-100 in 500 milli-second intervals, and it's output is displayed perfectly from the main app, coming with 500ms intervals.

Any ideas?

Answer 1

I've had the exact same problem, Calling a CLI app from C# .NET from which I needed to get realtime stdOut updates (progress report of an archiver), but it didn't send any output until after the process exited (Not helpful to get a progress report after 10 minutes of compressing)

From what I understand it is the process's fault, not flushing the stdOut buffer. I couldn't find any way on .Net to manually tell the process to flush its stdOut. However, I found a hacky solution to be able to read the stdOut in realtime.

All I did was access the Process.StandardOutput.BaseStream.ReadByte() method which does work and returns actual bytes sent to the stdOut. This is pure data including carriage manipulation etc.

Then convert the byte (seems to be ASCII?) to a character with Char.ConvertFromUtf32(..) and push the character into a stringBuilder object. And now, since I have stdOut data, I can deal with it however I want.

Example: For my case, I wanted to capture the stdOut word by word and then callback to user to handle each word.

// - Standard process
proc = new Process();
proc.StartInfo.RedirectStandardOutput = true;
proc.StartInfo.CreateNoWindow = true;
proc.StartInfo.filename = "...";
proc.start();

// - Init some things
int byte_r; // The byte that is going to be read from stdOut
StringBuilder word = new StringBuilder(); // Append the characters here
Action<String> onStdOutWord; // USERSET from before. Callbacks words read from stdOut

// As long as there is stdOut Data
while( (byte_r = proc.StandardOutput.BaseStream.ReadByte()) > -1 )
{   
    // If SPACE or ENTER callback the current word
    if(byte_r==32 || byte_r==13) {
        if(word.Length>0) {
            onStdOutWord(word.ToString());
            word.Clear();
        }
    }else{
        // Append character to string, skip special characters
        if(byte_r>32) {
            word.Append(Char.ConvertFromUtf32(byte_r));
        }
    }//-
}// - end while

Then, using the custom callback onStdOutWord() I was getting realtime data from the CLI app and handled it to get what I wanted.

The above is code just a use case since I wanted to get words, you can alter it and have it work like however you want. e.g. Not splitting into words and pushing out the entire stringBuilder object?

I know it's 8 years after the original question here, but I spend too much time researching on why I couldn't get stdout data like the OP, and figured to share my solution in case anyone else has the same problem and stumbles upon this page, like I did