Capturing bash script output from python code

Question

Im trying to execute a bash script through python, capture the output of the bash script and use it in my python code. Im using subprocess.run(), however, my output comes empty. Can you spot a mistake in my code?

when trying to forward the output to a file I can see the output currectly

Here is my python code - example.py (I'm using Python 3.8.5):


import os, subprocess

res = subprocess.run(os.path.join('T:', 'myfolder', 'input-script.sh'), shell=True, capture_output=True, text=True)

print(res)

Here is my bash script - input-script.sh:


#!/bin/sh

read -p "enter input:" reply

echo "output: $reply"


Bash input: "hello world"

Python output:

CompletedProcess(args='T:myfolder\\input-script.sh', returncode=0, stdout='', stderr='')

Expected Python output:

CompletedProcess(args='T:myfolder\\input-script.sh', returncode=0, stdout='hello world', stderr='')

Python already has `input()`, so why need bash? Also, appears you are on Windows host, so how would bash scripts run in the default CMD from `os.subprocess`? — OneCricketeer, Apr 10 '23 at 09:03
Are you sure `T:myfolder\\input-script.sh` is a correct path? It looks at the very least brittle. — tripleee, Apr 18 '23 at 06:59

mrCatlost · Answer 1 · 2023-04-11T13:49:57.667

First of all, looking to your code it would work correctly. But it would be great to know what are your version of python because that can change some type of attributes.

So an acclaration about the argument textthat you are using

Uses text=True will capture the STDOUT OR STDERR as str and if you don´t use it as bytes
text=True is used for Python >= 3.7 only, use universal_newlines=Trueon Python <= 3.6 (That maybe is your problem)

So this code would work correctly as you wrote it.

import os, subprocess

#< python 3.7

res = subprocess.run(os.path.join('T:', 'myfolder', 'input-script.sh'), shell=True, capture_output=True, universal_newlines=True)

print(res)

#>= python 3.7

res = subprocess.run(os.path.join('T:', 'myfolder', 'input-script.sh'), shell=True, capture_output=True, text=True)

print(res)

Anyway another way to get the result, is using subprocess.PIPE to capture STDOUT and STDERR independently. This can be used in any version of Python.

import os, subprocess

#>= python 3.7

res = subprocess.run(os.path.join('T:', 'myfolder', 'input-script.sh'), shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)

print(res.stdout)
print(res.stderr)

Tried to capture STDOUT and STDERR independently - didn't work — Ory, Apr 18 '23 at 06:18

score 0 · Answer 2 · answered Apr 18 '23 at 06:57

The output from subprocess.run is the CompletedProcessObject, not the actual output.

You want to capture the output with capture_output=True and access the correct attribute of res.

There is no need for shell=True here, assuming that the shebang is set up correctly etc. (I have no idea how exactly this works on Windows, though. If you need to run cmd in order to run Bash, you'll have to put it back.) Perhaps see also Actual meaning of shell=True in subprocess

res = subprocess.run(os.path.join('T:', 'myfolder', 'input-script.sh'), capture_output=True, text=True, check=True)
print(res.stdout)

This presupposes that the fundamental process creation works correctly on your system (i.e. you have installed Bash in the first place, etc etc). The added check=True instructs Python to raise an error if the process could not be run correctly; you should generally use this (but understand also that some processes will appear to raise an error even when they completed correctly; this would happen, for example, when grep could not find any matches, or diff found differences.)

For much much _much_ more, see also https://stackoverflow.com/questions/4256107/running-bash-commands-in-python/51950538#51950538 — tripleee, Apr 18 '23 at 06:57

Capturing bash script output from python code

2 Answers2