python subprocess output redirection

Question

I'm working on a python script that ssh into a server and read some file contents. Let's say they are file1.txt, file2.txt and file3.txt.

For now I'm doing

commands = "ssh {host} {port}; cat file1.txt; cat file2.txt; cat file3.txt"

p = subprocess.run(commands, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)

But this gives contents of three files concatenated. Is there some way that I can redirect each command's output to different pipe, so that I can easily distinguish which output came from which file?

Answer 1

When using subprocess the way you did, you are executing the commands in sequence and in a batch, while grabbing the stdout. My suggestion would be to run them one by one, e.g.:

command1 = "ssh {host} {port}; cat file1.txt"
command1 = "ssh {host} {port}; cat file2.txt"
command1 = "ssh {host} {port}; cat file3.txt"

commands = [command1, command2, command3]

for cmd in commands:
     p = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
     # DO SOMETHING ABOUT THE OUTPUT OR IN BETWEEN OUTPUTS

You could also establish an SSH connection and send commands one by one instead of logging in each time. For that I suggest that you look at this answer.