3

I have to files, main.py and child.py.

I am trying to send a string to the stdin of main.py.

This is my incomplete code:

main.py

from subprocess import *
import time

def main():
    program = Popen(['python.exe'. 'child.py', 'start'])
    while True: #waiting for'1' to be sent to the stdin
        if sys.stdin == '1':
            print('text)

if __name__ == '__main__':
    main()

child.py

import sys

if sys.argv[1] == 'start':
    inp = input('Do you want to send the argument?\n').lower()
    if inp == 'no':
        sys.exit()
    elif inp == 'yes':
        #Somehow send '1' to the stdin of 1.py while it is running

I have no idea how to do this.

I am running windows 10 with python 3.5.1

-Thanks

EDIT: When I am sending the argument back to main.py, I can not re-open the program. os.system re-opens the program which is not useful in my case.

These programs are a small demo of what I am trying to do. In my actual program, I am not able to do that as the two programs are "communicating" with each other an need to be open at all times.

What I need answered is a way to send an argument to main.py perhaps using stdin but when I am sending my argument, It can not re-open the program. Some examples like os.system re-open the program which is not what I am trying to do. I need main.py open at all times.

I have my new current code which is not working. A window pops up and then closes.

main.py

from subprocess import Popen, PIPE, STDOUT
x = Popen(['python.exe', '2.py', 'start'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
while x.poll() is None:
    if b'Do you want to send the argument?' in x.stdout.read():
        x.stdin.write(b'yes\n')

child.py

import sys
import time
time.sleep(1)
if 1 = 1:
    inp = input('Do you want to send the argument?\n').lower()
    if inp == 'no':
        sys.exit()
    elif inp == 'yes':
        sys.stdout.write('1')
        sys.stdout.flush()

That is my code.

Nic
  • 1,549
  • 2
  • 13
  • 13
  • Why not just do `import child`? – Torxed Jun 01 '16 at 05:58
  • Generic question: why do you want to do this? What is the context? –  Jun 01 '16 at 06:01
  • @VikasMadhusudana No, it's not: `Popen` already does that here. It's about passing stdout to the stdin of another program (in this case the parent), which in *nix parliance is shell related, not really Python. –  Jun 01 '16 at 06:03
  • The way I'm using this program, I cant just import child. @Torxed – Nic Jun 01 '16 at 06:03
  • The way main.py and child.py looks, yes you can because child would inherit `sys.argv` from the parent and there's also other ways to do regular python imports with custom arguments and global Variables. – Torxed Jun 01 '16 at 06:05
  • Possible duplicate of [Call python script with input with in a python script using subprocess](http://stackoverflow.com/questions/30076185/call-python-script-with-input-with-in-a-python-script-using-subprocess) – shivsn Jun 01 '16 at 06:06
  • Also, this sounds like a [XY problem](http://meta.stackexchange.com/q/66377/329549) – Torxed Jun 01 '16 at 06:07
  • No @VikasMadhusudana The way it is used in the "duplicate" is reopening the program, I don't want the program to be re opened, I instead want it to stay open. – Nic Jun 01 '16 at 06:07
  • @Torxed These programs are a small demo of what I am trying to do. In my actual program, I am not able to do that as the two programs are "communicating" with each other an need to be open at all times. – Nic Jun 01 '16 at 06:14
  • @Evert I have kind of explained a bit more on why I need it in my answer. – Nic Jun 01 '16 at 06:16
  • As I said, XY problem where you've made up a solution to your own problem but is having issues getting that solution to work as well, so instead of describing the actual problem and scenario you've managed to confuse us as well. Please update your question so that this information comes across clearly! – Torxed Jun 01 '16 at 06:18
  • From the last sentence in your updated question, I understand you want two Python programs to communicate with each other. Using stdout/stdin is *not* the way to do this. On *nix, I'd have said, use sockets. On Windows, something similar probably exists. You may want to search for something like "windows two programs communicate" or similar. See also Torxed comment above this comment. –  Jun 01 '16 at 06:20
  • @Torxed Have I made it a bit more clear? – Nic Jun 01 '16 at 06:22

1 Answers1

9

What you need is something along the lines of (in main.py):

from subprocess import Popen, PIPE, STDOUT
x = Popen(['some_child.exe', 'parameter'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)

while x.poll() is None:
    child_output = x.stdout.readline()
    print(child_output)
    if b'Do you want to send the argument?' in child_output:
        x.stdin.write(b'yes\n')
        x.stdin.flush()
x.stdout.close()
x.stdin.close()

You're assuming child.exe (in your mockup demo, python.exe) is communicating with main.py via sys.stdin/stdout, however these I/O's are used to communicate with the shell that spawned the process.

Much like the childs stdout/stdin will be communicating with the shell that spawned that process, in this case Popen().

Each spawned child process of subprocess.Popen(...) will be isolated with it's own stdout/stdin/stderr, otherwise every subprocess would make a huge mess of your main process stdout/stdin. This means you'll have to check for output on that particular subprocess and write to it accordingly as done in the above example.

One way to look at it is this: enter image description here

You're starting main.py, and you communicate with it via sys.stdout and sys.stdin. Each input() in main.py will output something to sys.stdout so you can read it.

Exactly the same logic applies to child.exe where every input() will output something to it's sys.stdout (- But remember - sys is not a shared variable across processes).

import sys

if sys.argv[1] == 'start':
    inp = input('Do you want to send the argument?\n').lower()
    if inp == 'no':
        sys.exit()
    elif inp == 'yes':
        #Somehow send '1' to the stdin of 1.py while it is running
        sys.stdout.write('1')
        sys.stdout.flush()

But a simple print(1) would do the same because it will essentially output the 1 to sys.stdout for you.

Edit 2018: Don't forget to close your inputs and outputs, as they might leave open file descriptors on your file system, hogging resources and causing problems later in life.

Other conveyers of information

Assuming you have control of the code to child.exe and you can modify the communication pipe in any way, some other options are:

More cautionary tails!

  • .readline() will assume there's a \n somewhere in your data, most likely at the end. I switched to .readline() for two reasons, .read() will hang and wait for EOF unless you specify exactly how many bytes to read, if I'm not out on a bicycle. To be able to read all kinds of output you need to incorporate select.select() into your code - or a buffer of some sort where you call x.stdout.read(1) to read one byte at a time. Because if you try to read .read(1024) and there's not 1024 bytes in the buffer, your read will hang until there are 1024 characters.

  • I left a bug in your child.py code on purpose (mine works) - It's trivial and basic Python - in hopes that it's a learning experience on how to debug errors (you mentioned you're not good at it, this is a way to learn).

Torxed
  • 22,866
  • 14
  • 82
  • 131
  • Okay, I kind of get what you mean. Could you give an example with using the stdout? @Torxed – Nic Jun 01 '16 at 06:26
  • @nic there's an example at the top of my answer? It's not perfect because I'm coding on me phone. I can perfect the code in a sec. – Torxed Jun 01 '16 at 06:28
  • okay, can you give the example with both the two files code? The reason why I say anoth example is because child.py has no variable "x" as it dose not have main.py open as a variable "x". Thanks – Nic Jun 01 '16 at 06:32
  • @Torxed, will buffering cause any issues? – gaganso Jun 01 '16 at 06:49
  • 1
    @SilentMonk Depending on what you mean with buffering in this case. Yes, if you do not empty the buffer of `x.stdout`, buffering will hang the process - There for it's extremely important to empty the IO buffer. – Torxed Jun 01 '16 at 06:59
  • @Nic I'm assuming you're a beginner? If so it might be worth mentioning. My example code should work out of the box with your `child.py`, however I think you've missunderstood the concepts of `sys.stdout` and what `stdout`is in the scope of a `subprocess.Popen` - So I did my best to explain it in more detail. – Torxed Jun 01 '16 at 07:00
  • Two things, Firstly, when I run it with python.exe in front of child.py (you need to for a python file) I have to remove the stdout = PIPE ect. Secondly, I get an error: if b'Do you want to send the argument?' in x.stdout.read(): AttributeError: 'NoneType' object has no attribute 'read' – Nic Jun 01 '16 at 07:07
  • 1
    @SilentMonk and yes, if the full output is not complete but in fact divided mid way my if statement won't catch the expected output. Kept it simple tho because OP is already confused about the IO. Should mention it tho. – Torxed Jun 01 '16 at 07:07
  • And also, thanks for all your help. All of the explaining really helped – Nic Jun 01 '16 at 07:08
  • @Nic - if you're going to insist on how I should answer your question - I suggest you answer it yourself. Because no `stdout = PIPE` in this case must stay, it's extremely important. Why have you gotten this idea? Stop asking XY problems. – Torxed Jun 01 '16 at 07:09
  • Well, If I dont include the 'python.exe' at the start, the python program will not work with a Win32 error. I am a beginner to python and to stack overflow with asking questions. – Nic Jun 01 '16 at 07:11
  • @Nic - Yes you should have `python.exe` first, but you said you - quote - *"I have to remove the `stdout = PIPE` etc"* - end quote - And that didn't make sense from what I just wrote as an answer. `['some_child.exe', 'parameter']` was a generic example to emphasis that you should execute the child there. In your case that would be `python.exe` **just as I wrote just a row later**. To be extremely super clear and over explaining this, you should do `Popen(['python.exe'. 'child.py', 'start'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)`. – Torxed Jun 01 '16 at 07:39
  • @Torxed When I run main.py in the shell, a window comes up and then closes and the program stops. I think it's from child.py closing really quickly. I am a massive beginner and I don't know why this is happening. – Nic Jun 01 '16 at 09:10
  • @Nic - Could you explain exactly how you start your `main.py`? Are you opening a new `cmd.exe` and manually entering `python.exe main.py`? And it still closes? – Torxed Jun 01 '16 at 09:23
  • @Torxed So, I copied you code into each of the files and replaced 'some_child.exe' with 'python.exe', child.py and 'parameter' with 'start'. I open up main.py to edit it with IDLE and run the program (F5). Once run, a window opens up and then closes. When opened up with python.exe by double clicking on the main.py, the same happens. Again, I am using python 3.5.1. – Nic Jun 01 '16 at 09:28
  • @Nic - Ok there's a **lot** of details missing here in your initial question. a IDLE is not Python, and every IDLE is different and sets up it's own environment variables, paths etc. Is `python.exe` in your windows `PATH` variable? If so, please run your script manually because I seriously have no idea how to debug a IDLE, especially one without a name. Do the following: `start menu` -> `cmd.exe` -> `python.exe main.py start` and see what the error output is. – Torxed Jun 01 '16 at 10:32
  • @Torxed sure although I can't at the moment, I did try just soluble clicking the program and only one window opened up. I'll be able to check it tomorrow. For me it's 20:36 so I'll be able to look at about 17:00. Tommorow and tell you what happens then. – Nic Jun 01 '16 at 10:37
  • @Nic Perfect, will wait. – Torxed Jun 01 '16 at 10:38
  • Btw @Torxed thx for all your help! When you say you will wait, 21 hours is a while. Anyway, see Ya then – Nic Jun 01 '16 at 10:40
  • @Torxed when i run the program, it creates a new blank line and then goes back to the cmd terminal where i can type cmd commands – Nic Jun 02 '16 at 09:08
  • @Nic Please post both your main and child code on for instance https://gist.github.com/ – Torxed Jun 02 '16 at 09:12
  • @Torxed should I post it as an answer so you can see? – Nic Jun 02 '16 at 09:13
  • @Torxed Or should I edit my question with my current code. – Nic Jun 02 '16 at 09:15
  • @Nic Here is fine. No need to update the question yet. – Torxed Jun 02 '16 at 09:17
  • @Nic Humm simple solution, I didn't assume you actually wanted to *see* the output of the child, just respond to it automatically. Updated my answer to do a `print()` on the childs output. – Torxed Jun 02 '16 at 09:24
  • @Torxed so where do I add the print()? What line. – Nic Jun 02 '16 at 09:26
  • @Nic I'm not in the best of moods at this particular moment, so I'll ask politely if you've even reviewed the change in my answer at the absolute top of the answer? There's literally only **one print** in the entire 5 lines code - And you can check it against your own version of main.py. You literally have to start trying things out for your self. I'm not saying this to be rude but programming is less about asking questions and hoping someone else has the answer for it and more about try things out until you either break it so bad you can't understand what you did wrong or you succeed. Sorta. – Torxed Jun 02 '16 at 09:31
  • Sorry, didn't see that there. As I am still a beginner, I'm not very good at fixing errors. When main.py opened, the python.exe window opens and no output is displayed. – Nic Jun 02 '16 at 09:38
  • @Nic If you simply write (in your cmd.exe): `python.exe` and hit enter, what happens? And if you do `python.exe --version` what pops out? – Torxed Jun 02 '16 at 09:50
  • @Torxed Firstly, python.exe dosent work, I need to type py. When py is typed, it brings up the python idle in the cmd. When I type py --version, I get Python 3.5.1 – Nic Jun 02 '16 at 10:15
  • @Torxed I think the reason that the python window opens up is that it is the first argument in my Popen although I need it as I thought the child.py would open through it. If I do not include the python.exe, I get a win32 error with the child.py not being an executable. – Nic Jun 02 '16 at 10:17
  • @Nic Ok so I didn't quote get what you said, but you probably want to try `Popen(['py', 'child.py', 'start'] ...)` as your Popen statement. Or to be extra certain that this is default Windows friendly, do `Popen(['C:\\Program Files\\Python3\\python.exe', 'child.py', 'start'] ...)`. You should locate your `python.exe` and use an absolute path. – Torxed Jun 02 '16 at 10:24
  • @Torxed I tried that but it didn't work ALTHOUGH, if I run the program from IDLE(F5), python still opens up but when closed, it displays the text from the child.py – Nic Jun 02 '16 at 10:27
  • @Nic Yea nic, this isn't related to much programming anymore - I'm more than certain that this is 100% related to your non-standard installation of Python. Here's the code above - and it works: http://i.imgur.com/uHYGjM6.png. You can ignore the fact that I'm running Linux since if your python executable is in your system path, this should work. Please find your `python.exe` and use the absolute path and your code should work. – Torxed Jun 02 '16 at 10:31
  • @Torxed I'll try re installing python – Nic Jun 02 '16 at 10:36
  • @Torxed I think I know the problem. This is my new code for testing: https://shrib.com/rFzKbCBEFtLiOAa?v=nc When I run main.py, it waits for 3 seconds like in the code(time.sleep(1) x 3) and then prints out: b'continue in 3\r\n2\r\n1\r\nhey\r\ntest' somthing happened I think its somthing to do with main.py not displaying the stdout. – Nic Jun 02 '16 at 10:51
  • @Nic Congratulations, the code works. Every `print()` will be picked up by `main.py` and it's `Popen()` handle. That's the intention. – Torxed Jun 02 '16 at 10:53
  • @Torxed What I wanted it to do was print 'continue in 3' and then wait 1 second then print '2'. Not for it all to be in one go at the end. – Nic Jun 02 '16 at 10:54
  • @Nic - That's a totally different question not subject under this topic if you ask me. This is related to how you pick up stdout/stdin of a subprocess, which you now can. What you're asking now is a matter of your choice of code and how to make is so that it behaves the way you'd like it to. Something I'd suggest you play around with before opening a new question as well (because people want some form or proof that you've tried to solve it yourself first before helping out) – Torxed Jun 02 '16 at 11:00
  • @Nic Pleasure is mine, sorry if I sound grumpy at all. I usually insert more smily faces in my comments, didn't feel like it today tho : ) Best of luck and maybe we'll see each other around on SO. If my answer solves your needs - feel free to mark it as a accepted answer (or if someone comes with a better one, mark that one) so we tidy up the pile of "unsolved questions" here on SO : ) – Torxed Jun 02 '16 at 11:03