Communicating between 2 python scripts using popen

Question

Let's say we got the problem as displayed in the figure.

scriptA.py                               scriptB.py

get directory
modify directory
run scriptB with directory as input
                                -------> get directory
                                         open .txt file
                                         scan its contents
                                <------- return content to scriptA
print content

My ScriptA example is:

import os
import subprocess
    
if __name__ == '__main__':
   directory = os.getcwd() + os.sep + 'some extension'
    
   p = subprocess.Popen("python ScriptB.py", stdin=subprocess.PIPE, stdout=subprocess.PIPE, shell=True)
        
   results = p.communicate(input = str.encode(directory))[0]
    
   print(results)

But now I'm not sure how to access the input in ScriptB. And what is necessary so communicate detects the content which is the output of ScriptB? And is my example correct?

Why not import the other script as a module, or is that for an exercise? — bereal, Apr 26 '21 at 07:35
I had to do something not too dissimilar recently, was disappointed/frustrated by `subprocess`, and ended up using `pexpect` instead: https://pexpect.readthedocs.io/en/stable/ — mechanical_meat, Apr 26 '21 at 07:42

tripleee · Answer 1 · 2021-04-26T07:58:02.920

As noted in comments, a vastly superior solution to what you propose is usually to refactor scriptB so that you can import it from scriptA and just call its functions directly, without requiring a separate subprocess.

If you can't or don't want to do that, the absolutely simplest arrangement is to write scriptB so that it accepts a command-line parameter, and prints the result to standard output.

results = subprocess.run(
    ['python', 'scriptB.py', directory],
    check=True, text=True, capture_output=True).stdout

Notice the absence of shell=True, which adds no value at all here, and the corresponding division of the command line into a list of strings. Perhaps see also Actual meaning of 'shell=True' in subprocess

If you need scriptB.py to accept input on standard input (often, a dubious design for many reasons) that's not very different;

result = subprocess.run(
    ['python', 'scriptB.py'],
    input=directory + '\n',
    check=True, text=True, capture_output=True).stdout

You really want to avoid subprocess.Popen() unless you are in a situation which subprocess.run() or its owner siblings cannot handle.

Communicating between 2 python scripts using popen

1 Answers1