Bash scripting from python

Question

I am confused about invoking bash from python code.
I wrote a small script that shows strange behaviors when invoking bash from python :

import subprocess

def check_output ( command,**w ):
    print( '\n>>> %s'%command )
    try :
        return subprocess.check_output( command,**w ).decode()
    except Exception as e :
        return str( e )

print( check_output( ['ls','-l']))
print( check_output( ['ls','-l',';','ls','-a']))
print( check_output( 'ls -l' ))
print( check_output( 'ls -l ; ls -a' ))
print( check_output( ['echo','99'],shell=1))
print( check_output( ['echo','11',';','echo','22'],shell=1))
print( check_output( 'echo 33 ; echo 44',shell=1 ))

And the output:

>>> ['ls', '-l']
total 124
-rwxrwxrwx 1 root root    59 Mar 22 09:57 main.py
-rwxrwxrwx 1 root root  3885 Mar 17 17:30 README.md
drwxrwxrwx 0 root root  4096 Mar 22 10:33 src
-rwxrwxrwx 1 root root    36 Mar 22 09:57 sub.c
drwxrwxrwx 0 root root  4096 Mar 22 10:33 test


>>> ['ls', '-l', ';', 'ls', '-a']
ls: cannot access ';': No such file or directory
ls: cannot access 'ls': No such file or directory
Command '['ls', '-l', ';', 'ls', '-a']' returned non-zero exit status 2.

>>> ls -l
[Errno 2] No such file or directory: 'ls -l': 'ls -l'

>>> ls -l ; ls -a
[Errno 2] No such file or directory: 'ls -l ; ls -a': 'ls -l ; ls -a'

>>> ['echo', '99']

>>> ['echo', '11', ';', 'echo', '22']

>>> echo 33 ; echo 44
33
44

The first and last commands are as I expected, but the others seems strange to me.
The multiple commands separator ; seems to act differently depending on parameter shell. And also the concatenated commands...

By the way I get the same results with python 2 and 3 (under kali linux).

Note that using `shell=True` is generally a bad idea. First, but less importantly, it's inefficient: Why start a copy of `sh` and tell _it_ to start `gcc`, when you could instead just start `gcc` directly? Second, and more importantly, it raises security risks: When a shell is involved you need to worry if some joker told your program to save the compiled file as `$(rm -rf ~).o`; but without a shell, shell syntax in your data is harmless. — Charles Duffy, Mar 18 '21 at 18:30
As for your question, though -- the real issue is identical to that in [can a shell script set environment variables of the calling shell?](https://stackoverflow.com/questions/496702/can-a-shell-script-set-environment-variables-of-the-calling-shell); the difference (that there the parent process is a shell and here it's a Python interpreter) is immaterial to the problem. — Charles Duffy, Mar 18 '21 at 18:34

score 1 · Accepted Answer · answered Mar 18 '21 at 18:27

When you launch a new process, that process gets a copy of your current environment, but any changes that it makes are only to its copy. When the process ends, that environment is thrown away. What that means is you can say in Python:

os.environ{'LD_LIBRARY_PATH'] = "/home/user/lib"

and all of the processes you start from then on will inherit that variable. But EACH copy of execute starts over with a fresh copy of your environment. Whatever changes they make do not survive.

Bash scripting from python

1 Answers1