StandardOutput.ReadToEnd() hangs

Question

I have a program that frequently uses an external program and reads its outputs. It works pretty well using your usual process redirect output, but one specific argument for some reason hangs when I try to read it, no error message - no exception, it just 'stops' when it reaches that line. I of course use a centralized function to call and read output from the program, which is this:

public string ADBShell(string adbInput)
{
    try
    {
        //Create Empty values
        string result = string.Empty;
        string error = string.Empty;
        string output = string.Empty;
        System.Diagnostics.ProcessStartInfo procStartInfo 
            = new System.Diagnostics.ProcessStartInfo(toolPath + "adb.exe");

        procStartInfo.Arguments = adbInput;
        procStartInfo.RedirectStandardOutput = true;
        procStartInfo.RedirectStandardError = true;
        procStartInfo.UseShellExecute = false;
        procStartInfo.CreateNoWindow = true;
        procStartInfo.WorkingDirectory = toolPath;
        System.Diagnostics.Process proc = new System.Diagnostics.Process();
        proc.StartInfo = procStartInfo;
        proc.Start();
        // Get the output into a string
        proc.WaitForExit();
        result = proc.StandardOutput.ReadToEnd();
        error = proc.StandardError.ReadToEnd();  //Some ADB outputs use this
        if (result.Length > 1)
        {
            output += result;
        }
        if (error.Length > 1)
        {
            output += error;
        }
        Return output;
    }
    catch (Exception objException)
    {
        throw objException;
    }
}

The line that hangs is result = proc.StandardOutput.ReadToEnd();, but again, not every time, only when sent a specific argument ("start-server"). All other arguments work just fine - it reads the value and returns it. It's also strange the way it hangs. It doesn't freeze or give an error or anything, it just stops processing. As if it was a 'return' command, except it doesn't even return to the calling function, it just stops everything with the interface still up and running. Anyone experienced this before? Anyone have any idea what I should try? I'm assuming it's something unexpected within the stream itself, but is there a way I can handle/ignore this so that it reads it anyway?

Answer 1

Proposed solutions with BeginOutputReadLine() are a good way but in situations such as that, it is not applicable, because process (certainly with using WaitForExit()) exits earlier than async output finished completely.

So, I tried to implement it synchronously and found that the solution is in using Peek() method from StreamReader class. I added check for Peek() > -1 to sure that it is not the end of the stream as in MSDN article described and finally it works and stop hanging!

Here is the code:

var process = new Process();
process.StartInfo.CreateNoWindow = true;
process.StartInfo.UseShellExecute = false;
process.StartInfo.RedirectStandardOutput = true;
process.StartInfo.RedirectStandardError = true;
process.StartInfo.WorkingDirectory = @"C:\test\";
process.StartInfo.FileName = "test.exe";
process.StartInfo.Arguments = "your arguments here";

process.Start();
var output = new List<string>();

while (process.StandardOutput.Peek() > -1)
{
    output.Add(process.StandardOutput.ReadLine());
}

while (process.StandardError.Peek() > -1)
{
    output.Add(process.StandardError.ReadLine());
}
process.WaitForExit();

Answer 2

The problem is that you are using the synchronous ReadToEnd methods on both the StandardOutput and the StandardError streams. This can lead to a potential deadlock you are experiencing. This is even described in the MSDN. The solution is described there. Basically, it is: Use the asynchronous version BeginOutputReadLine to read the data of the StandardOutput stream:

p.BeginOutputReadLine();
string error = p.StandardError.ReadToEnd();
p.WaitForExit();

Implementation of Async reading using BeginOutputReadLine see in ProcessStartInfo hanging on "WaitForExit"? Why?

Answer 3

What about something like:

process.Start();
process.BeginOutputReadLine();
process.BeginErrorReadLine();

process.OutputDataReceived += (sender, args) =>
{
    var outputData = args.Data;
    // ...
};
process.ErrorDataReceived += (sender, args) =>
{
    var errorData = args.Data;
    // ...
};
process.WaitForExit();

Answer 4

I had the same kind of problem that error was just hanging.

Based on your response to Daniel Hilgarth I didn't even try using those codes though i think they would have worked for me.

Since I want to be able do some fancier output still eventually i decided that I would do it with both of the outputs being done in a background thread.

public static class RunCommands
{
    #region Outputs Property

    private static object _outputsLockObject;
    private static object OutputsLockObject
    { 
        get
        {
            if (_outputsLockObject == null)
                Interlocked.CompareExchange(ref _outputsLockObject, new object(), null);
            return _outputsLockObject;
        }
    }

    private static Dictionary<object, CommandOutput> _outputs;
    private static Dictionary<object, CommandOutput> Outputs
    {
        get
        {
            if (_outputs != null)
                return _outputs;

            lock (OutputsLockObject)
            {
                _outputs = new Dictionary<object, CommandOutput>();
            }
            return _outputs;
        }
    }

    #endregion

    public static string GetCommandOutputSimple(ProcessStartInfo info, bool returnErrorIfPopulated = true)
    {
        // Redirect the output stream of the child process.
        info.UseShellExecute = false;
        info.CreateNoWindow = true;
        info.RedirectStandardOutput = true;
        info.RedirectStandardError = true;
        var process = new Process();
        process.StartInfo = info;
        process.ErrorDataReceived += ErrorDataHandler;
        process.OutputDataReceived += OutputDataHandler;

        var output = new CommandOutput();
        Outputs.Add(process, output);

        process.Start();

        process.BeginErrorReadLine();
        process.BeginOutputReadLine();

        // Wait for the process to finish reading from error and output before it is finished
        process.WaitForExit();

        process.ErrorDataReceived -= ErrorDataHandler;
        process.OutputDataReceived -= OutputDataHandler;

        Outputs.Remove(process);

        if (returnErrorIfPopulated && (!String.IsNullOrWhiteSpace(output.Error)))
        {
            return output.Error.TrimEnd('\n');
        }

        return output.Output.TrimEnd('\n');
    }

    private static void ErrorDataHandler(object sendingProcess, DataReceivedEventArgs errLine)
    {
        if (errLine.Data == null)
            return;

        if (!Outputs.ContainsKey(sendingProcess))
            return;

        var commandOutput = Outputs[sendingProcess];

        commandOutput.Error = commandOutput.Error + errLine.Data + "\n";
    }

    private static void OutputDataHandler(object sendingProcess, DataReceivedEventArgs outputLine)
    {
        if (outputLine.Data == null)
            return;

        if (!Outputs.ContainsKey(sendingProcess))
            return;

        var commandOutput = Outputs[sendingProcess];

        commandOutput.Output = commandOutput.Output + outputLine.Data + "\n";
    }
}
public class CommandOutput
{
    public string Error { get; set; }
    public string Output { get; set; }

    public CommandOutput()
    {
        Error = "";
        Output = "";
    }
}

This worked for me and allowed me to not have to use a timeout for the read.

Answer 5

I had the same deadlock problem. This code snippet worked for me.

ProcessStartInfo startInfo = new ProcessStartInfo("cmd")
{
    WindowStyle = ProcessWindowStyle.Hidden,
    UseShellExecute = false,
    RedirectStandardInput = true,
    RedirectStandardOutput = true,
    CreateNoWindow = true
};

Process process = new Process();
process.StartInfo = startInfo;
process.Start();
process.StandardInput.WriteLine("echo hi");
process.StandardInput.WriteLine("exit");
var output = process.StandardOutput.ReadToEnd();
process.Dispose();

Answer 6

Something that is elegant and worked for me is:

Process nslookup = new Process()
{
   StartInfo = new ProcessStartInfo("nslookup")
   {
      RedirectStandardInput = true,
      RedirectStandardOutput = true,
      UseShellExecute = false,
      CreateNoWindow = true,
      WindowStyle = ProcessWindowStyle.Hidden
   }
};

nslookup.Start();
nslookup.StandardInput.WriteLine("set type=srv");
nslookup.StandardInput.WriteLine("_ldap._tcp.domain.local"); 

nslookup.StandardInput.Flush();
nslookup.StandardInput.Close();

string output = nslookup.StandardOutput.ReadToEnd();

nslookup.WaitForExit();
nslookup.Close();

This answer I found here and the trick is using Flush() and Close() on standard input.

Answer 7

The accepted answer's solution didn't work for me. I had to use tasks in order to avoid the deadlock:

//Code to start process here

String outputResult = GetStreamOutput(process.StandardOutput);
String errorResult = GetStreamOutput(process.StandardError);

process.WaitForExit();

With a GetStreamOutput function as follows:

private string GetStreamOutput(StreamReader stream)
{
   //Read output in separate task to avoid deadlocks
   var outputReadTask = Task.Run(() => stream.ReadToEnd());

   return outputReadTask.Result;
}

Answer 8

Just in case someone stumbles upon this question while wiling to use Windows Forms and TextBox (or RichTextBox) to show the errors and outputs the process returns in real time (as they are written to process.StandardOutput / process.StandardError).

You need to use OutputDataReceived() / ErrorDataReceived() in order to read both streams without deadlocks, there is no way (as far as I know) to avoid deadlocks otherwise, even Fedor's answer, which now holds the "Answer" tag and the most likes up to date, does not do the trick for me.

However, when you use the RichTextBox (or TextBox) to output the data, another problem you encounter is how to actually write the data into the textbox in real time (once it arrives). You receive the access to the data inside one of the background threads OutputDataReceived() / ErrorDataReceived() and you can only AppendText() from the main thread.

What I first tried doing was calling process.Start() from a background thread and then calling BeginInvoke() => AppendText() in OutputDataReceived() / ErrorDataReceived() threads while the main thread was process.WaitForExit().

However, this led to my form freezing and ultimately hanging for eternity. After a few days of trying I ended up with the solution below, that seems to work pretty well.

Shortly speaking, you need to add the messages into a concurrent collection inside OutputDataReceived() / ErrorDataReceived() threads while the main thread should constantly try to extract messages from that collection and append them into the textbox:

            ProcessStartInfo startInfo
                = new ProcessStartInfo(File, mysqldumpCommand);

            process.StartInfo.FileName = File;
            process.StartInfo.Arguments = mysqldumpCommand;
            process.StartInfo.CreateNoWindow = true;
            process.StartInfo.UseShellExecute = false;
            process.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
            process.StartInfo.RedirectStandardInput = false;
            process.StartInfo.RedirectStandardError = true;
            process.StartInfo.RedirectStandardOutput = true;
            process.StartInfo.StandardErrorEncoding = Encoding.UTF8;
            process.StartInfo.StandardOutputEncoding = Encoding.UTF8;
            process.EnableRaisingEvents = true;

            ConcurrentQueue<string> messages = new ConcurrentQueue<string>();

            process.ErrorDataReceived += (object se, DataReceivedEventArgs ar) =>
            {
                string data = ar.Data;
                if (!string.IsNullOrWhiteSpace(data))
                    messages.Enqueue(data);
            };
            process.OutputDataReceived += (object se, DataReceivedEventArgs ar) =>
            {
                string data = ar.Data;
                if (!string.IsNullOrWhiteSpace(data))
                    messages.Enqueue(data);
            };

            process.Start();
            process.BeginErrorReadLine();
            process.BeginOutputReadLine();
            while (!process.HasExited)
            {
                string data = null;
                if (messages.TryDequeue(out data))
                    UpdateOutputText(data, tbOutput);
                Thread.Sleep(5);
            }

            process.WaitForExit();

The only downside of this approach is the fact that you can loose messages in a quite rare case, when process starts writing them between process.Start() and process.BeginErrorReadLine() / process.BeginOutputReadLine(), just keep that in mind. The only way to avoid that is to read the full streams and (or) gain access to them only when the process finishes.

Answer 9

first

     // Start the child process.
     Process p = new Process();
     // Redirect the output stream of the child process.
     p.StartInfo.UseShellExecute = false;
     p.StartInfo.RedirectStandardOutput = true;
     p.StartInfo.FileName = "Write500Lines.exe";
     p.Start();
     // Do not wait for the child process to exit before
     // reading to the end of its redirected stream.
     // p.WaitForExit();
     // Read the output stream first and then wait.
     string output = p.StandardOutput.ReadToEnd();
     p.WaitForExit();

second

 // Do not perform a synchronous read to the end of both 
 // redirected streams.
 // string output = p.StandardOutput.ReadToEnd();
 // string error = p.StandardError.ReadToEnd();
 // p.WaitForExit();
 // Use asynchronous read operations on at least one of the streams.
 p.BeginOutputReadLine();
 string error = p.StandardError.ReadToEnd();
 p.WaitForExit();

This is from MSDN