How to asynchronously read the standard output stream and standard error stream at once

Question

I want to read the ouput of a process in the form as is in a console (standard output is blended with standard error in one stream). Is there a way how to do it?

I was thinking about using

ProcessStartInfo.UseShellExecute = true;  

but then I cannot read asynchronously the output. If I set

process.ProcessStartInfo.UseShellExecute = false;  
process.StartInfo.RedirectStandardOutput = true;
process.OutputDataReceived += new DataReceivedEventHandler(partialOutputHandler);

then I can read standard output (I can do the same for standard error) but I don't know how to simulate the behavior of console (the blending of stdout and stderr).

This is similar to Linux which has the feature of redirecting standard error stream to the standard output stream; how?

Answer 1

Do you mean something like this?

SynchronizationContext _syncContext;
MyForm()
{
    _syncContext = SynchronizationContext.Current;
}

void StartProcess()
{
    using (var process = new Process
        {
            StartInfo = new ProcessStartInfo
            {
                FileName = "myProcess.exe",
                UseShellExecute = false,
                RedirectStandardOutput = true,
                RedirectStandardError = true,
            }
        })
    {
        process.OutputDataReceived += (sender, args) => Display(args.Data);
        process.ErrorDataReceived += (sender, args) => Display(args.Data);

        process.Start();
        process.BeginOutputReadLine();
        process.BeginErrorReadLine();

        process.WaitForExit(); //you need this in order to flush the output buffer
    }   
}

void Display(string output)
{
    _syncContext.Post(_ => myTextBox.AppendText(output), null);
}

Answer 2

The MSDN article states:

The redirected StandardError stream can be read synchronously or asynchronously. Methods such as Read, ReadLine, and ReadToEnd perform synchronous read operations on the error output stream of the process. These synchronous read operations do not complete until the associated Process writes to its StandardError stream, or closes the stream.

In contrast, BeginErrorReadLine starts asynchronous read operations on the StandardError stream. This method enables a designated event handler for the stream output and immediately returns to the caller, which can perform other work while the stream output is directed to the event handler.

Synchronous read operations introduce a dependency between the caller reading from the StandardError stream and the child process writing to that stream. These dependencies can result in deadlock conditions. When the caller reads from the redirected stream of a child process, it is dependent on the child. The caller waits on the read operation until the child writes to the stream or closes the stream. When the child process writes enough data to fill its redirected stream, it is dependent on the parent. The child process waits on the next write operation until the parent reads from the full stream or closes the stream. The deadlock condition results when the caller and child process wait on each other to complete an operation, and neither can proceed. You can avoid deadlocks by evaluating dependencies between the caller and child process.

The same applies to the StandardOutput, so you just read both streams asynchronously.

Merging both streams into one complicates detection of what output is the error reporting and what is the 'product' information.

Answer 3

I found the answer:

The output streams are buffered. There is no way to get the true sequential order of the items inserted into the streams. In fact it makes little sense as both streams can be written too at the same time. They are independent of each other. Therefore the best you can do is get the sequential output from each one as they arrive.

Generally this is not an issue though as almost all console apps use standard output for both output and error messages. The error stream is used by some apps but the messages are generally duplicates of the errors generated in the output stream.

Source: http://social.msdn.microsoft.com/Forums/uk/csharpgeneral/thread/192b6df7-9437-42cf-81c1-c125021735ba

Answer 4

Similar kind of example, with exception that I'm collecting stdout and error into separate strings using StringBuilder for that purpose.

/// <summary>
/// Executes command
/// </summary>
/// <param name="cmd">command to be executed</param>
/// <param name="output">output which application produced</param>
/// <param name="transferEnvVars">true - if retain PATH environment variable from executed command</param>
/// <returns>true if process exited with code 0</returns>
static bool ExecCmd(string cmd, out String output, bool transferEnvVars = false)
{
    ProcessStartInfo processInfo;
    Process process;

    if (transferEnvVars)
        cmd =  cmd + " && echo --VARS-- && set";

    processInfo = new ProcessStartInfo("cmd.exe", "/c " + cmd);
    processInfo.CreateNoWindow = true;
    processInfo.UseShellExecute = false;
    processInfo.RedirectStandardError = true;
    processInfo.RedirectStandardOutput = true;

    process = Process.Start(processInfo);

    // Executing long lasting operation in batch file will hang the process, as it will wait standard output / error pipes to be processed.
    // We process these pipes here asynchronously.
    StringBuilder so = new StringBuilder();
    process.OutputDataReceived += (sender, args) => { so.AppendLine(args.Data); };
    StringBuilder se = new StringBuilder();
    process.ErrorDataReceived += (sender, args) => { se.AppendLine(args.Data); };

    process.BeginOutputReadLine();
    process.BeginErrorReadLine();
    process.WaitForExit();

    output = so.ToString();
    String error = se.ToString();

    if (transferEnvVars)
    {
        Regex r = new Regex("--VARS--(.*)", RegexOptions.Singleline);
        var m = r.Match(output);
        if (m.Success)
        {
            output = r.Replace(output, "");

            foreach ( Match m2 in new Regex("(.*?)=([^\r]*)", RegexOptions.Multiline).Matches(m.Groups[1].ToString()) )
            {
                String key = m2.Groups[1].Value;
                String value = m2.Groups[2].Value;
                Environment.SetEnvironmentVariable(key, value);
            }
        }
    }

    if(error.Length != 0)
        output += error;
    int exitCode = process.ExitCode;

    if (exitCode != 0)
        Console.WriteLine("Error: " + output + "\r\n" + error);

    process.Close();
    return exitCode == 0;
}