63

I'm using the subprocess.Popen call, and in another question I found out that I had been misunderstanding how Python was generating arguments for the command line.

My Question
Is there a way to find out what the actual command line was?

Example Code :-

proc = subprocess.popen(....)
print "the commandline is %s" % proc.getCommandLine()

How would you write getCommandLine ?

pradyunsg
  • 18,287
  • 11
  • 43
  • 96
Brian Postow
  • 11,709
  • 17
  • 81
  • 125

5 Answers5

91

It depends on the version of Python you are using. In Python3.3, the arg is saved in proc.args:

proc = subprocess.Popen(....)
print("the commandline is {}".format(proc.args))

In Python2.7, the args not saved, it is just passed on to other functions like _execute_child. So, in that case, the best way to get the command line is to save it when you have it:

proc = subprocess.Popen(shlex.split(cmd))
print "the commandline is %s" % cmd

Note that if you have the list of arguments (such as the type of thing returned by shlex.split(cmd), then you can recover the command-line string, cmd using the undocumented function subprocess.list2cmdline:

In [14]: import subprocess

In [15]: import shlex

In [16]: cmd = 'foo -a -b --bar baz'

In [17]: shlex.split(cmd)
Out[17]: ['foo', '-a', '-b', '--bar', 'baz']

In [18]: subprocess.list2cmdline(['foo', '-a', '-b', '--bar', 'baz'])
Out[19]: 'foo -a -b --bar baz'
unutbu
  • 842,883
  • 184
  • 1,785
  • 1,677
  • I'm in 2.6. And in 2.6 at least it's good that list2cmdline is undocumented, because it doesn't work: for '--arg=foo bar' what python ACTUALLY does is '--arg="foo bar"' but what list2cmdline gives is '"--arg=foo bar"'... but thanks. – Brian Postow Feb 12 '13 at 16:49
  • I think the problem there is not with `list2cmdline`, but rather `shlex.split`. `shlex.split('--arg="foo bar"')` returns a list with a single item: `['--arg=foo bar']`, while `shlex.split('--arg = "foo bar"')` correctly splits the arguments: `['--arg', '=', 'foo bar']`. On this latter list `list2cmdline` works fine. – unutbu Feb 12 '13 at 16:58
  • By the way, Python2.6's `subprocess` module uses `list2cmdline` to convert `args` to a list whenever `args` is not a string so it is working in the sense that what you are seeing as the return value of `list2cmdline(args)` is exactly what is being passed on to `execvp*` or the Windows equivalent. – unutbu Feb 12 '13 at 17:04
  • When I pass "--arg=foo bar" to popen, it works (ie, it passes --arg="foo bar" to the program). When I pass the exact same string to list2cmdline, it gives "--arg=foo bar" (with quotation marks) – Brian Postow Feb 12 '13 at 17:05
  • I'm confused about what you are actually passing to `list2cmdline`. It expects a list not a string. – unutbu Feb 12 '13 at 17:10
  • Sorry, set `cmdline = ["cmd", "--arg=foo bar", "baz"]` I then run popen with cmdline, and get `cmd --arg="foo bar" baz` When I call `list2cmdline(cmdline)` I get `cmd "--arg=foo bar" baz` – Brian Postow Feb 12 '13 at 17:21
  • Are you saying that when you use `subprocess.Popen('cmd "--arg=foo bar" baz', shell=True)` it works, but when you use `subprocess.Popen(shlex.split('cmd "--arg=foo bar" baz'))` (without `shell=True`) it fails? – unutbu Feb 12 '13 at 17:29
  • No. I have cmdline as a list, as above. I call `subprocess.Popen(cmdline)` and it works. When I call `list2cmdline(cmdline)` it gives a commandline that will not work. – Brian Postow Feb 12 '13 at 18:00
  • Could you post a runnable example? (I am unable to reproduce this problem.) – unutbu Feb 12 '13 at 18:03
  • Well I'd need to write a program that prints its arguments out, otherwise you don't see what the program is actually GETTING... Let me see what I can whip up – Brian Postow Feb 12 '13 at 19:33
  • 1
    AHA. So the issue is that my program is adding the quotes. the thing that was confusing to me was that Popen never actually CREATES a command line. Everything is done through IPC, therefore if one of the arguments in the list is '--arg=foo bar' then that's fine, that's an element in the list. there is no need for quotes. Then on the RECEIVING program's end, it adds the quotes because it needs them for other reasons, and it expects the shell to have stripped off the ones that were there initially... – Brian Postow Feb 12 '13 at 19:58
7

The correct answer to my question is actually that there IS no command line. The point of subprocess is that it does everything through IPC. The list2cmdline does as close as can be expected, but in reality the best thing to do is look at the "args" list, and just know that that will be argv in the called program.

Brian Postow
  • 11,709
  • 17
  • 81
  • 125
  • 4
    [`list2cmdline()`](https://hg.python.org/cpython/file/f38489a3394f/Lib/subprocess.py#l541) is useful only on Windows for applications compatible with how MS C runtime parses command-line. It converts a list of arguments to a string that is passed to `CreateProcess()`. [`cmd.exe` uses different rules](http://stackoverflow.com/q/27864103/4279). On POSIX the list is passed directly to `os.execve()` (+/- os.fsencode()). – jfs Feb 05 '15 at 16:55
2

Beautiful and scalable method

I have been using something like this:

#!/usr/bin/env python3

import os
import shlex
import subprocess
import sys

def run_cmd(cmd, cwd=None, extra_env=None, extra_paths=None, dry_run=False):
    if extra_env is None:
        extra_env = {}
    newline_separator = ' \\\n'
    out = []
    kwargs = {}
    env = os.environ.copy()

    # cwd
    if 'cwd' is not None:
        kwargs['cwd'] = cwd

    # extra_env
    env.update(extra_env)
    for key in extra_env:
        out.append('{}={}'.format(shlex.quote(key), shlex.quote(extra_env[key])) + newline_separator)

    # extra_paths
    if extra_paths is not None:
        path = ':'.join(extra_paths)
        if 'PATH' in env:
            path += ':' + env['PATH']
        env['PATH'] = path
        out.append('PATH="{}:${{PATH}}"'.format(':'.join(extra_paths)) + newline_separator)

    # Command itself.
    for arg in cmd:
        out.append(shlex.quote(arg) + newline_separator)

    # Print and run.
    kwargs['env'] = env
    print('+ ' + '  '.join(out) + ';')
    if not dry_run:
        subprocess.check_call(cmd, **kwargs)

run_cmd(
    sys.argv[1:],
    cwd='/bin',
    extra_env={'ASDF': 'QW ER'},
    extra_paths=['/some/path1', '/some/path2']
)

Sample run:

./a.py echo 'a b' 'c d'

Output:

+ ASDF='QW ER' \
  PATH="/some/path1:/some/path2:${PATH}" \
  echo \
  'a b' \
  'c d' \
;
a b c d

Feature summary:

  • makes huge command lines readable with one option per line
  • add a + to commands like sh -x so users can differentiate commands from their output easily
  • show cd, and extra environment variables if they are given to the command. These only printed if given, generating a minimal shell command.

All of this allows users to easily copy the commands manually to run them if something fails, or to see what is going on.

Tested on Python 3.5.2, Ubuntu 16.04. GitHub upstream.

Ciro Santilli OurBigBook.com
  • 347,512
  • 102
  • 1,199
  • 985
0

You can see it by passing the process id to ps command, if you are on POSIX OS:

import subprocess

proc = subprocess.Popen(["ls", "-la"])
subprocess.Popen(["ps", "-p", str(proc.pid)])

Output (see the CMD column):

  PID TTY           TIME CMD
 7778 ttys004    0:00.01 ls -la
catwith
  • 875
  • 10
  • 13
0

On windows, I used @catwith 's trick (thanks, btw):

wmic process where "name like '%mycmd%'" get processid,commandline

where "mycmd" is a part of the cmd unique to your command (used to filter irrelevant system commands)

That's how I revealed another bug in the suprocess vs windows saga. One of the arguments I had had its double-quotes escaped a-la unix! \"asdasd\"

Berry Tsakala
  • 15,313
  • 12
  • 57
  • 80