3

Right now I have a test file.dat that I run hexdump on and put the output into a hexdump.dat file. 

subprocess.Popen(['hexdump file.dat > hexdump.dat' ], shell=True)

As a side note, I have seen suggestions to not use shell=True but I essentially get the error OSError: [Errno 2] No such file or directory.

So, I would like to be able to pass in a variable or an array, files, instead of the hardcoded "file.dat". "files" could be a user input or an array/list generated from a previous subprocess section.

I have tried a user input case:

from subprocess import Popen, PIPE, STDOUT 
files = raw_input('File Name: ')                                                                                                 
p = subprocess.Popen(['hexdump files > hexdump.dat' ], stdout=PIPE, stderr=STDOUT)                                         
out,err = p.communicate(input=files)

Also with:

p = subprocess.Popen(['hexdump',  inputs, ' > hexdump.dat' ], stdout=PIPE, stderr=STDOUT)

Thanks for the help, I know I'm not appropriately understanding the structure needed here so some "handholdy" answers would be appreciated.

Pierre Barre
  • 2,174
  • 1
  • 11
  • 23
stackofpancakes
  • 53
  • 1
  • 9
  • What is the issue with what you tried ? – Pierre Barre Apr 27 '16 at 21:31
  • Error Output :Traceback (most recent call last): File "scratchpadDecoder.py", line 47, in subprocess.Popen(['hexdump file.dat > hexdump.dat' ]) File "/dsw/python-2.7.1/lib/python2.7/subprocess.py", line 672, in __init__ errread, errwrite) File "/dsw/python-2.7.1/lib/python2.7/subprocess.py", line 1202, in _execute_child raise child_exception – stackofpancakes Apr 27 '16 at 21:35

5 Answers5

4

You need shell=True because otherwise it'll look for an executable with that name. shell=True tells the method to use the shell to execute the command so > and friends become what you originally intended them to be (redirection).

The following code you posted: 

from subprocess import Popen, PIPE, STDOUT 
files = raw_input('File Name: ')                                                                                                 
p = subprocess.Popen(['hexdump files > hexdump.dat' ], stdout=PIPE, stderr=STDOUT)                                         
out,err = p.communicate(input=files)

will not work because you're just passing files to hexdump, and if a file with the name files doesn't exist you'll get an error (and if it does exist, it's still probably not what you wanted.)

What you want is to build the string you're executing:

file = "input.dat"
p = subprocess.Popen("hexdump " + file + " > hexdump.dat", shell=True)
itdoesntwork
  • 4,666
  • 3
  • 25
  • 38
  • Wow thanks. I had thought I had tried the "+" combo in the past with no difference between commas but I guess that was for a different issue. Very pleased. – stackofpancakes Apr 27 '16 at 21:45
  • 1
    I'd also like to point out that while I was answering your question about variables, the code I've posted isn't really idiomatic. The answers that don't use shell=True and use the redirection provided by the subprocess module have better code. – itdoesntwork Apr 27 '16 at 21:50
3

Warning: Passing shell=True can be a security hazard if combined with untrusted input. See the warning under Frequently Used Arguments for details.

Something like:

with open('hexdump.dat', 'wb') as f:
    p = subprocess.Popen(['hexdump', 'file.dat'], stdout=f)
    p.wait()

You should read up on Popen and what the shell argument does, and make your decision.

totoro
  • 2,469
  • 2
  • 19
  • 23
1

Instead of redirecting with >, you can redirect using the stdout param. As for the list of files, you can just append the list of files to an array containing hexdump, ie

myfiles = ['file1','file2']
with open('hexdump.dat', 'w') as output:
    proc = subprocess.Popen(['hexdump'] + myfiles, stdout=output)
wonton
  • 7,568
  • 9
  • 56
  • 93
  • The only hiccup with this is that I think it puts all the hexdumps into one hexdump.dat and I still need them distinct for later usage. But I will see about incorporating this when I address the array issue. – stackofpancakes Apr 27 '16 at 21:48
  • what's the array issue? if you want to output multiple files, you'll need to use a loop or something to that effect – wonton Apr 27 '16 at 21:52
  • Unfortunately this results in the error : TypeError: can only concatenate list (not "str") to list – stackofpancakes Apr 27 '16 at 21:55
  • "array issue" as in once I build the back end for that, right now I'm testing this with an array of one test case – stackofpancakes Apr 27 '16 at 21:56
  • No reason for `communicate()`. There is no data to send or receive. – totoro Apr 27 '16 at 22:12
  • make sure it's ['hexdump'] + ['file1'] and not ['hexdump'] + 'file1' – wonton Apr 27 '16 at 22:22
1

I found the simplest way for me to do shell redirection with python and variables is as follows:

subprocess.check_output('svnadmin load %s < %s' % (repo, fname), shell=True)

It can handle very large files.

mauricio777
  • 1,296
  • 1
  • 15
  • 15
-1

First off, in regards to file not found, you may need to specify the current working directory. 

subprocess.Popen(['hexdump file.dat > hexdump.dat' ], shell=True, cwd='/bar/foo')

In regards to passing in an array as an argument, that is typically something like:

args = [ 'hexdump', ] + inputs
subprocess.Popen( args, cwd='/foo/bar' )
Clarus
  • 2,259
  • 16
  • 27