How can I use * in a subprocess inside a python script?

Question

I'm trying to delete a big number of folders in some code I am writing in Python 2.7, the folders to be deleted are in the same folder than the code I'm running. To do this I use subprocess in the following way:

import subprocess


def runsafe(job,args):
    jobs=[job];
    for arg in args:
        jobs.append(arg)
    proc=subprocess.Popen(jobs,stdout=subprocess.PIPE)
    return proc.stdout.readlines()


if __name__=="__main__":
    runsafe("rm",["-r","./*/"])

If I do this the code doesn't delete any folders. Obviously, if I write rm -r ./*/ in my terminal all the folders inside the folder where I'm running the code are deleted. Also if I change runsafe("rm",["-r","./*/"]) in my code for runsafe("rm",["-r","./foo/"]) (where foo is one of the folders to be deleted) the code works fine and deletes the desired folder.

How can I delete all the folders at once? Why doesn't it work if I use * to indicate that I want to delete all the folders?

score 4 · Accepted Answer · edited May 23 '17 at 12:23

4

You can't use * because subprocess doesn't run in a shell (e.g. Bash), and it is the shell that expands * into a list of files.

You could use shell=True to tell subprocess to run in a shell, but this isn't a great idea, and there are better alternatives.

If you need to recursively delete a directory you can use shutil.rmtree() and avoid starting a subprocess at all.

edited May 23 '17 at 12:23

Community

1
1

answered Feb 23 '16 at 14:47

dimo414

47,227
18
148
244

score 0 · Answer 2 · answered Feb 23 '16 at 14:33

0

You could have subprocess run a shell to interpret your command. The command line needs to be passed as a string, not a list:

subprocess.Popen('rm -r *', shell=True, ...)

answered Feb 23 '16 at 14:33

Thomas Lotze

5,153
1
16
16

2

But `shell=True` is discouraged, as its a security risk. Particularly when paired with an `rm -r` command, I'd be hesitant to do this. – dimo414 Feb 23 '16 at 14:45
I'd not see much of a problem as long as the command string is created programmatically. User (or other) input would be a real problem, though, even if sanitised. – Thomas Lotze Feb 23 '16 at 14:50
1

Even if it's "created programmatically" it's more error prone, since the shell then has to parse the string, meaning whitespace and other special characters can cause problems. – dimo414 Feb 23 '16 at 14:52
Yes, every moving part is one more opportunity for bugs to sneak in. I'd not consider it security relevant beyond the measure of security relevance every bug has in principle. – Thomas Lotze Feb 23 '16 at 14:54
1

`shell=True` is conceptually similar to constructing SQL queries manually (as opposed to using prepared statements). Sure, you can do it safely, but why take the risk? It is much more dangerous than any arbitrary bug; it risks giving a user shell access to your system. – dimo414 Feb 23 '16 at 14:59
1

I quite like the SQL analogy, and I completely understand your point. Mine, however, was that as long as user input doesn't get near the subprocess call, you may still be pointing a battery of interesting guns at your own foot by `shell=True`, but it's giving the user shell access no more than giving them "rm access" by calling `rm` directly. – Thomas Lotze Feb 23 '16 at 15:25
Fair enough, append "and common source of serious bugs" to the first sentence in my original comment. – dimo414 Feb 23 '16 at 15:33

How can I use * in a subprocess inside a python script?

2 Answers2