Python - subprocess and the env argument

Question

I'm finishing up with a project and there is only one thing left for me to do; looking at an elegant way to launching programs using subprocess including custom environment variables set.

cat script.py

#!/usr/bin/env python

import argparse
import subprocess
import sys
import os

parser = argparse.ArgumentParser()

parser.add_argument(
                    action='store',
                    nargs='*',
                    dest="foovar"
                    )

parser.add_argument('-a', action='store',
                    type=int,
                    default='43',
                    dest="var_version",
                    help='set the agent version to use. default:(4)')

parser.add_argument('-s', action='store',
                    type=int,
                    default='35',
                    dest="var_server",
                    help='set the version. default:(5)')

args = parser.parse_args()

sub_env = os.environ.copy()
for key in vars(args):
    if key.startswith('var_'):
        sub_env[key] = vars(args)[key]

print(vars(args))
print(args)   # Namespace(foovar=['james', 'marc'], var_version=43, var_server=35)

for machine in args.foovar:
        exit_code = subprocess.call("bash script.sh %s" % machine, env=sub_env, shell=True)

here the script.sh

#!/bin/bash

if [[ $(printenv | grep 'var_') ]]; then
        echo "var_* is working!"
fi

here the output :

$ ./script.py james marc -a 43 -s 35
{'var_version': 43, 'foovar': ['james', 'marc'], 'var_server': 35}
Namespace(foovar=['james', 'marc'], var_server=35, var_version=43)
Traceback (most recent call last):
  File "./script.py", line 44, in <module>
    exit_code = subprocess.call("bash script.sh %s" % machine, env=sub_env, shell=True)
  File "/usr/lib/python2.7/subprocess.py", line 523, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
TypeError: execve() arg 3 contains a non-string value

nb: this post has been updated following comment advice.

`var(args)['foovar']` is a rather ugly way to write `args.foovar`. — chepner, Feb 05 '20 at 15:09
`subprocess.Popen` returns a process, not an exit status. You need to manage the process yourself, or, if you just want to wait for the subprocess, switch to `subprocess.run` instead. You can also avoid the stupid `shell=True` by converting the string to a list: `['bash', 'script.sh', machine]`. Perhaps see also https://stackoverflow.com/questions/4256107/running-bash-commands-in-python/51950538#51950538 — tripleee, Feb 05 '20 at 15:11

Gilles 'SO- stop being evil' · Accepted Answer · 2020-02-05T15:06:28.490

1

Pass the env parameter to the subprocess function (Popen in your case). To modify the environment of the calling process, make a copy of os.environ. For example:

sub_env = os.environ.copy()
for key in vars(args):
    if key.startswith('var_'):
        sub_env[key] = args[key]
for machine in vars(args)['foovar']:
    exit_code = subprocess.Popen("…", env=sub_env, …)

edited Feb 05 '20 at 15:06

answered Feb 05 '20 at 12:19

Gilles 'SO- stop being evil'

104,111
38
209
254

I get a TypeError. File "./script.py", line 34, in for key in args: TypeError: 'Namespace' object is not iterable . – nerabis Feb 05 '20 at 14:04
@nerabis Oh right, it's a `Namespace` so you need `vars(args)` to iterate over it. Anyway that's just an example, the point is that you copy `os.environ` and add (or remove) stuff to it. – Gilles 'SO- stop being evil' Feb 05 '20 at 15:07
I edited the post - Thanks for your advice. not sure why am getting : `TypeError: execve() arg 3 contains a non-string value` – nerabis Feb 06 '20 at 21:40

score 1 · Answer 2 · answered Feb 06 '20 at 21:33

Your arguments var_version=43 and var_server=35 are Python integers. You need to convert them to strings to be able to pass them to a subprocess.

sub_env = os.environ.copy()
for key in vars(args):
    if key.startswith('var_'):
        sub_env[key] = str(getattr(args, key))

Python - subprocess and the env argument

2 Answers2