2

I'm trying to send data from a python script to my c# application using a standard input stream. I also need to eventually send data back from my c# application to the another python script. I first tried to do this with a UDP Connection which works fine for a couple lines, but I have a fair amount of data to send (a few thousand lines) and a requirement for data integrity which UDP cannot provide. I could also write to a file, but that seems very inefficient.

One last restriction is that while my two applications are related I cannot setup a direct connection between them using something like IronPython as they are both spawned separately by a 3rd party application.

This is what I am currently trying, but it is not working. Similar to this question: Passing data between Python and C# without writing a file

p = subprocess.Popen(C_SHARP_EXECUTABLE_FILE_PATH, stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
p.communicate(blob)

On my C# side I'm not entirely sure how to read this data, but I've tried using things like a loop around this: 

Console.ReadLine()

or getting the standard input and reading from it directly using:

Console.OpenStandardInput();

The current issue is that as soon as I call p.communicate my Python script gets locked and doesn't proceed. If it's waiting for the line to be read, what do I need to do to make it stop waiting? I tried only providing the stdin parameter, but that didn't help.

Community
  • 1
  • 1
Fozefy
  • 665
  • 2
  • 7
  • 23
  • How big is `blob`? Docs say there could be problems with large input. Does it hang when passing small message? Ctrl+D sends "end of input" signal on Unix, Ctrl+Z Enter closes stdin on Windows. – void Jan 06 '17 at 23:24
  • I've tested with a single line of a few characters and it still hangs. However I will eventually have a few thousands lines in the blob, is there a better mechanism for this? – Fozefy Jan 06 '17 at 23:35
  • What do you need is called [IPC](https://en.wikipedia.org/wiki/Inter-process_communication#Approaches). I would recommend trying [0MQ](http://zguide.zeromq.org/page:all#Ask-and-Ye-Shall-Receive), though you should be aware that IPC is generally more complex than working with stdin / stdout. – void Jan 08 '17 at 20:06
  • Is this going to be significantly faster then simply reading/writing to/from a file? That is very simple and I currently have it working, I'm not sure how much of an advantage this would be. Any thoughts? – Fozefy Jan 09 '17 at 03:51
  • That depends too much on applications using IPC and on infrastructure. With some skill one can achieve millions of messages per second passed by [ZeroMQ](http://zeromq.org/results:multicore-tests). And slow receiver can ensure bad performance even when taking data from files. Just do a quick smoke test. Real tasks which require outstanding IPC performance are rare. – void Jan 09 '17 at 16:40
  • Possible duplicate of [Passing data between Python and C# without writing a file](http://stackoverflow.com/questions/22180694/passing-data-between-python-and-c-sharp-without-writing-a-file) It turns out the the host application which was spawning my python and c# programs was blocking the pipe as it already had its own (mostly undocumented) way of passing data between the two applications. If I spawn two completely separate applications the linked code works just fine sending a few thousand lines. – Fozefy Jan 10 '17 at 20:57

0 Answers0