0

Trying to get the filenames using aws commands in python.But getting error.

import subprocess
output=subprocess.check_output(['aws', 's3', 'ls', 's3://dev-lake/staging/20181111/', '--recursive', '--human-readable', '--summarize', '|','awk', '{print $4}'])
print output

Error

Unknown options: |,awk,{print $4}
Traceback (most recent call last):
  File "list.py", line 2, in <module>
    output=subprocess.check_output(['aws', 's3', 'ls', 's3://sdx-dev-lake/staging/20181111/', '--recursive', '--human-readable', '--summarize', '|','awk', '{print $4}'])
  File "/usr/lib64/python2.7/subprocess.py", line 219, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['aws', 's3', 'ls', 's3://sdx-dev-lake/staging/20181111/', '--recursive', '--human-readable', '--summarize', '|', 'awk', '{print $4}']' returned non-zero exit status 255
marjun
  • 696
  • 5
  • 17
  • 30
  • please add the error message. it will help also. – MEdwin Nov 26 '18 at 15:18
  • It looks like your bash command is the problem. `subprocess.CalledProcessError: Command ... returned non-zero exit status 255` Does it work when you exec it from bash? – Raoslaw Szamszur Nov 26 '18 at 15:22
  • You do not have a shell, so you can't use pipes in your command eg. `cmd1 | cmd2`. Either use the option `shell=True` and pass the whole command as a single string or create the pipe in python using `Popen`. – Dunes Nov 26 '18 at 15:24
  • @Dunes That's not necessarily true you can use pipe for example with os module. `os.system("cmd1 | cmd2")` – Raoslaw Szamszur Nov 26 '18 at 15:29
  • @RaoslawSzamszur `os.system` executes the command in a subshell. It's semantically the same thing as using `shell=True`. – Dunes Nov 26 '18 at 15:36
  • @Dunes good to know, thanks! – Raoslaw Szamszur Nov 26 '18 at 16:02

0 Answers0