StreamReader readLine() hangs when reading in large line of JSON Output C#

Question

I'm currently using C# and the StreamReader Class to read in StandardOutput from a Python File which retrieves server data in a JSON format. The user can enter a search key (serverName, IP, ID, etc) and it would return this data below. The user can also search for as many server details as they please, sometimes up a thousand search keys.

Example:

[{'Search': 'serverName', 'Name': 'test.testing.com', 'IP': 
'10.XXX.XXX.X', 'State': 'OK', 'OS': 'Windows', 'ID':'123456'}]

I'm currently reading the output and adding it to a list within the C# code.

StreamWriter sw = pythonProcess.StandardInput;
StreamReader sr = pythonProcess.StandardOutput;

//Code to read in the search key inputs    

returnList.Add(sr.ReadLine());

The output is all contained within one line (not sure if that's the best way to handle it) so looping is not needed. It can read smaller outputs fine however, when if the user enters a large list of search keys, the program hangs on the sr.Readline() piece.

Whenever I run the Python Script in the terminal, I'm able to retrieve the JSON output fairly quickly no matter how large the search key array is. Is there a better/faster way to handle reading in a large line of JSON output from the stream without causing it to hang?

Thanks in advance

EDIT:

Here is my Python process information as well..

pythonProcess = new Process();
    try
    {
        pythonProcess.StartInfo.FileName = filename;
        pythonProcess.StartInfo.RedirectStandardInput = true;
        pythonProcess.StartInfo.RedirectStandardOutput = true;
        pythonProcess.StartInfo.RedirectStandardError = true;
        pythonProcess.StartInfo.Verb = "runas";
        pythonProcess.StartInfo.UseShellExecute = false;
        pythonProcess.StartInfo.Arguments = "-u \"" + serverPath + "\\" + scriptName + "\"";
        pythonProcess.Start(); // start the process (the python program)            
    }
    catch (Exception e)
    {

    }

Your line must be extremely and abnormally large for `StreamReader` to hang. I have never experienced such thing as `StreamReader` is pretty fast since it deals with a `Stream`. Can you please provide an example of your JSON String, then I can run some tests and can give you a proper solution. — Transcendent, Apr 12 '17 at 15:18
@rory.ap: Async won't solve the problem, it still takes too long, only that it prevents the app from freezing. I think the point is to speed up the process. — Transcendent, Apr 12 '17 at 15:19
@Transcendent -- Yeah I was a bit vague, but what I was suggesting implicitly was that 1.) there's no way to speed it up, i.e. it is what it is and 2.) you can keep the app from freezing in order to be more user-friendly. — rory.ap, Apr 12 '17 at 15:21
Reading output is non-trivial and many factors will cause either your process or the launched process to deadlock. See for example: [StandardOutput.ReadToEnd() hangs](http://stackoverflow.com/questions/7160187/standardoutput-readtoend-hangs), and [ProcessStartInfo hanging on “WaitForExit”? Why?](http://stackoverflow.com/questions/139593/processstartinfo-hanging-on-waitforexit-why). Read the comments on the accepted answers and you'll see that they aren't even flawless depending on the conditions, but there's about 20 different answers you can try in those questions. — Quantic, Apr 12 '17 at 15:31
@Transcendent -- It's similar to the output I specified above (ex. Search, IP, name, state) with their corresponding values but for thousands of servers all contained on one line which is read in with the StreamReader ReadLine() function. If that makes sense? — msleone, Apr 12 '17 at 15:32
@msleone: In that case the problem is not with the `StreamReader`, you should look for it elsewhere. For example `pythonProcess.StandardOutput`. — Transcendent, Apr 12 '17 at 15:33
@Transcendent made a change to the question and added my process code if that helps. — msleone, Apr 12 '17 at 16:58

StreamReader readLine() hangs when reading in large line of JSON Output C#

0 Answers0