How to use Unix regex character with python3 subprocesses run?

Question

I'm trying to get the output from a shell command in a Python3 program. I've been looking at the documentation and this is what I've been working with:

sortered=subprocess.run(
    # ['sort', time.strftime("%Y")+'*', '|', 'uniq', '-u'],  # causes error did not understand * char
    #['ls', '-l'],  # worked fine
    ['sort', '2019*'],  # gives same error as before
    capture_output=True
)

After running the script I get back this error:

$ myscript.py
CompletedProcess(args=['sort', '2019*'], returncode=2, stdout=b'', stderr=b"sort: cannot read: '2019*': No such file or directory\n")
To_Downloadtest1.sh has been created successfully

If I run the command normally using * it works fine.

$ sort 2019*
a file.
a file.
this one is a.
this one is b.

The script is ran from the same directory that the files starting with 2019 are in.

.
├── 2019-A.txt
├── 2019-B.txt
└── myscript.py

What should happen when I run the python script is the output from the command should be put into a variable as a string. This is not happening. The only time I get an error from subprocess.run comes from using * in the command. Otherwise, I get the stdout from subprocess.run correctly. I tried ls -l as a test and it worked correctly. How can I use * with subprocess.run?

Shelling out to `sort` not necessarily the best way to go about sorting a list of filenames in Python. — Tomalak, Oct 06 '19 at 12:08
@Tomalak `sort` sorts the *contents* of the files, not the names. — tripleee, Oct 06 '19 at 12:11
Same difference. Python can read the contents of files, Python can sort strings. There is no need to rely on the shell for that or introduce the overhead of child proceses. — Tomalak, Oct 06 '19 at 12:14

score 0 · Accepted Answer · answered Oct 06 '19 at 12:07

The problem here is that you don't have a shell, and so the wildcard does not get expanded.

You can use

sortered = subprocess.run('sort 2019*', shell=True, capture_output=True)

or

import glob
sortered = subprocess.run(['sort'] + glob.glob('2019*'), capture_output=True)

or, of course,

import glob

lines = []
for file in glob.glob('2019*') as handle:
    lines.extend(handle.read())
sortered = sorted(lines)

How to use Unix regex character with python3 subprocesses run?

1 Answers1