How to use linux command in subprocess python

Question

I have linux command like below:

 find /data/*/hr/ -printf "%f: %p: %u: %g %m (%M) \n"

How do i use in python subprocess check_output

I have tried like below but not working

 file_name = "/data/*/%s/" % (filename)
 get_perm = check_output(["find", file_name, "-printf", '\"%f: %p: %u: %g %m (%M) \n\"'])

Error I am getting:

find: ‘/data/*/hr/’: No such file or directory
Traceback (most recent call last):
  File "handover.py", line 90, in <module>
    get_perm = check_output(["find", file_name, "-printf", '\"%f: %p: %u: %g %m (%M) \n\"'])
  File "/usr/lib64/python2.7/subprocess.py", line 573, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['find', '/data/*/hr/', '-printf', '"%f: %p: %u: %g %m (%M) \n"']' returned non-zero exit status 1

Do you get any output? Do you get an error (i.e. a `CalledProcessError`?). If it dumped because it didn't like the `-printf"%f: %p:...` argument, it should have raised something — Nick T, Sep 10 '17 at 23:40
Python is also translating `\n` in the string into a literal newline, which might be strange when `find` gets it. You can also avoid escaping `"` chars by using `'` chars to wrap the string literal in. — Nick T, Sep 10 '17 at 23:42
Possible duplicate of [Running shell command from Python and capturing the output](https://stackoverflow.com/questions/4760215/running-shell-command-from-python-and-capturing-the-output) — dlmeetei, Sep 11 '17 at 02:55

score 2 · Answer 1 · answered Sep 11 '17 at 05:35

2

Finally,

I found below method

cmd = "find /data/*/{}/* -printf \"%f:%p:%u:%g:%m\n\"".format(filename)
info = subprocess.Popen(cmd,stdout=subprocess.PIPE,shell=True)
print info.stdout.read()

This solves my problem

answered Sep 11 '17 at 05:35

gopinara

51
8

Peter Kirby · Answer 2 · 2017-09-11T03:57:26.710

You are getting this error because the specified file does not exist. If you run the command directly in a shell, you will get the same response.

For example, the following works as expected:

import subprocess
import os

file_name = os.path.join(os.getcwd(), 'test.txt')
with open(file_name, 'w') as f:
    f.write('hello world')

get_perm = subprocess.check_output([
    "find",
     file_name,
     "-printf",
    '"%f: %p: %u: %g %m (%M) \n"'
    ], shell=True)

print(get_perm)
os.remove(file_name)

According to the docs:

If the return code [from subprocess.check_output] was non-zero it raises a CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute and any output in the output attribute.

I recommend you wrap your check_output call in a try..except, and catch the CalledProcessError.

Alternatively, if you really don't want to deal with the exception, you could instead execute the command:

x=$(find ~/data/*/hr/ -printf "%f: %p: %u: %g %m (%M) \n" 2>/dev/null || true) && echo $x

This will never return nonzero and will only ever contain output if the file exists.

Edit As Michael pointed out, the '*' is not getting expanded. However, if you set shell=True, it will. Try modifying your command as follows:

 file_name = "/data/*/%s/" % (filename)
 get_perm = check_output(["find", file_name, "-printf", '"%f: %p: %u: %g %m (%M) \n"'], shell=True)

While it's true that the file isn't found, the problem is likely that the `*` in the path should be expanded. — Michael Mior, Sep 11 '17 at 02:16
@MichaelMior Good catch. If you set `shell=True`, it will expand that the splat. — Peter Kirby, Sep 11 '17 at 04:01

score 0 · Answer 3 · answered Sep 11 '17 at 02:15

0

Your shell will expand the * in /data/*/hr/ when you call on the command line. Calling the function directly via check_output causes find to look for literally the directory /data/*/hr/. You could use the glob module to expand the path before passing to find:

import glob

file_name = "/data/*/%s/" % (filename)
get_perm = check_output(["find"] + glob.glob(file_name) + ["-printf", '\"%f: %p: %u: %g %m (%M) \n\"'])

glob.glob simply produces an array of path names which match the given expression by expanding any *s and some other special characters.

answered Sep 11 '17 at 02:15

Michael Mior

28,107
9
89
113

Hi Michael, It works fine.. But problem is filename "/data/*/fin/" . If fin does not exist. It will list other values. If fin does not exit. I expect to get no such file or directory error – gopinara Sep 11 '17 at 04:24
@gopinara It will list other files because you have a trailing slash, which the `find` command interprets as you indicating you want to list all of the files/directories contained within /fin/. Try removing the trailing slash. You can also specify a type to find to limit the results to only files or directories, like, `find /data/*/fin -type f`. Also, the first argument to find is usually the directory which you want to search within for a file or directory. Maybe you want something like `find /data/*/fin -type f -name your_file_name.ext` – Peter Kirby Sep 11 '17 at 04:40
I tried removing slash and adding -type d..same result – gopinara Sep 11 '17 at 05:04
You may just want to check if `glob.glob(file_name)` produces an empty array and if so, don't run the command at all. – Michael Mior Feb 09 '18 at 03:29

How to use linux command in subprocess python

3 Answers3