0

I am trying to run a shell script using through Python using subprocess.Popen().

The shell script just has the following lines:

#!/bin/sh
echo Hello World

Following is the Python code:

print("RUNNNING SHELL SCRIPT NOW")
shellscript = subprocess.Popen(['km/example/example1/source/test.sh'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
shellscript.wait()

for line in shellscript.stdout.readlines():
    print(line)
print("SHELL SCRIPT RUN ENDED")

However, on running this, I am only getting the following output:

RUNNNING SHELL SCRIPT NOW
SHELL SCRIPT RUN ENDED

i.e. I am not getting the shell script output in between these 2 lines.

Moreover, when I remove the stderr=subprocess.PIPE part from the subprocess, I get the following output:

RUNNNING SHELL SCRIPT NOW
'km' is not defined as an internal or external command.
SHELL SCRIPT RUN ENDED

I am not able to understand how to resolve this, and run the shell script properly. Kindly guide. Thanks.

UPDATE:

I also tried the following change:

print("RUNNNING SHELL SCRIPT NOW")
shellscript = subprocess.Popen(['km/example/example1/source/test.sh'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)

out, err = shellscript.communicate()
print(out)
print("SHELL SCRIPT RUN ENDED")

I get the following output:

RUNNNING SHELL SCRIPT NOW
b''
SHELL SCRIPT RUN ENDED
tripleee
  • 175,061
  • 34
  • 275
  • 318
eraisedtothepowerx
  • 49
  • 1
  • 7
  • maybe [this](https://stackoverflow.com/questions/1388753/how-to-get-output-from-subprocess-popen-proc-stdout-readline-blocks-no-dat) and [that](https://stackoverflow.com/questions/2502833/store-output-of-subprocess-popen-call-in-a-string) will help you – jsofri Nov 23 '21 at 06:48
  • Usually the first to do when using subprocess, is to check if `shellscript.returncode == 0` otherwise there as was an error (details should be available in `err`). The command passed to `Popen` has to be a string when using `shell=True`, whereas `shell=False` requires the command as `list` – Maurice Meyer Nov 23 '21 at 07:23
  • You are using a relative path (not starting with `/`). Nothing wrong with that *provided you make sure what the current working directory is*. When using an IDE it is often not what you would expect. (Just add `import os` and `print(os.getcwd)` to control it...) – Serge Ballesta Nov 23 '21 at 07:31
  • use shell=True and pass the command as string not list and one more thing your path is not correct provide absolute path to the script from very begining e.g. /var/www/file.html – Zain Ul Abidin Nov 23 '21 at 07:31
  • 1
    @ZainUlAbidin That is _precisely_ incorrect advice. You want to avoid `shell=True` when you can. – tripleee Nov 23 '21 at 09:27
  • @tripleee Yes i know but i don't think so for now he even care about security he just wants his code to be executed and show some output – Zain Ul Abidin Nov 23 '21 at 09:32

1 Answers1

2

The simple and straightforward fix is to not use bare Popen for this.

You also don't need a shell to run a subprocess; if the subprocess is a shell script, that subprocess itself will be a shell, but you don't need the help of the shell to run that script.

proc = subprocess.run(
    ['km/example/example1/source/test.sh'],
    check=True, capture_output=True, text=True)
out = proc.stdout

If you really need to use Popen, you need to understand its processing model. But if you are just trying to get the job done, the simple answer is don't use Popen.

The error message actually looks like you are on Windows, and it tries to run km via cmd which thinks the slashes are option separators, not directory separators. Removing the shell=True avoids this complication, and just starts a process with the requested name. (This of course still requires that the file exists in the relative file name you are specifying. Perhaps see also What exactly is current working directory? and also perhaps switch to native Windows backslashes, with an r'...' string to prevent Python from trying to interpret the backslashes.)

tripleee
  • 175,061
  • 34
  • 275
  • 318
  • Thank you for the answer. Yes I don't really need to use Popen. Yes, I am on Windows. When I run your code snippet, I get ` OSError: [WinError 193] %1 is not a valid Win32 application ` What could this be related to? – eraisedtothepowerx Nov 24 '21 at 04:21
  • If `km/example/example1/source/test.sh` is not a valid Windows application, that would be the explanation. I don't know how you would expect shell scripts to work on Cygwin; maybe explicitly run it with `['/bin/sh', 'km/example/example1/source/test.sh']`? – tripleee Nov 24 '21 at 06:51
  • I had to provide the complete path starting from the Drive. However, when the script runs, a terminal window open for just a second displays the output and closes itself. Is there a way to display the output on the Pycharm terminal instead, so that it stays there? – eraisedtothepowerx Nov 25 '21 at 02:47
  • No experience with PyCharm and very little of Python on Windows; but this sounds like a common FAQ. Ask a new question if you can't find it already answered. – tripleee Nov 25 '21 at 07:00