OSError: [Errno 8] Exec format error

Question

I am having hard time parsing the arguments to subprocess.Popen. I am trying to execute a script on my Unix server. The script syntax when running on shell prompt is as follows: /usr/local/bin/script hostname = <hostname> -p LONGLIST. No matter how I try, the script is not running inside subprocess.Popen

The space before and after "=" is mandatory.

import subprocess
Out = subprocess.Popen(['/usr/local/bin/script', 'hostname = ', 'actual server name', '-p', 'LONGLIST'],shell=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE)

The above does not work.

And when I use shell=False, I get OSError: [Errno 8] Exec format error

Wild guess: try `'hostname = actual server name'` instead of `'hostname = ', 'actual server name'` — Kevin, Dec 22 '14 at 16:43
Based on how you say you run it at the prompt, it looks like `hostname` is one argument and `=` is a separate argument, which is quite strange. Are you sure there are spaces around `=`? — Bryan Oakley, Dec 22 '14 at 16:45
hi Bryan, yes there has to be space. Script accepts key=value type argument. — user3477108, Dec 22 '14 at 17:13
@user3477108 - that's puzzling, you say there has to be a space but then give a "key=value" example that doesn't have a space. Of course, since your command line example has a space, we know that `key = value` should be 3 arguments to `Popen` as insti shows. — tdelaney, Dec 22 '14 at 17:47
Thanks. I wrote a little wrapper script around my original script to disregard space around "=" The wrapper script runs fine. — user3477108, Dec 22 '14 at 18:11
do you see the shebang line (`#!` -- the first two bytes in the file) at the top of `/usr/local/bin/script` ? — jfs, Dec 24 '14 at 16:32
@user3477108: I've updated [my answer](http://stackoverflow.com/a/27608363/4279) to explain and show how to fix `Exec format error` — jfs, Dec 28 '14 at 09:24

jfs · Answer 1 · 2014-12-28T09:24:03.113

OSError: [Errno 8] Exec format error can happen if there is no shebang line at the top of the shell script and you are trying to execute the script directly. Here's an example that reproduces the issue:

>>> with open('a','w') as f: f.write('exit 0') # create the script
... 
>>> import os
>>> os.chmod('a', 0b111101101) # rwxr-xr-x make it executable                       
>>> os.execl('./a', './a')     # execute it                                            
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/os.py", line 312, in execl
    execv(file, args)
OSError: [Errno 8] Exec format error

To fix it, just add the shebang e.g., if it is a shell script; prepend #!/bin/sh at the top of your script:

>>> with open('a','w') as f: f.write('#!/bin/sh\nexit 0')
... 
>>> os.execl('./a', './a')

It executes exit 0 without any errors.

On POSIX systems, shell parses the command line i.e., your script won't see spaces around = e.g., if script is:

#!/usr/bin/env python
import sys
print(sys.argv)

then running it in the shell:

$ /usr/local/bin/script hostname = '<hostname>' -p LONGLIST

produces:

['/usr/local/bin/script', 'hostname', '=', '<hostname>', '-p', 'LONGLIST']

Note: no spaces around '='. I've added quotes around <hostname> to escape the redirection metacharacters <>.

To emulate the shell command in Python, run:

from subprocess import check_call

cmd = ['/usr/local/bin/script', 'hostname', '=', '<hostname>', '-p', 'LONGLIST']
check_call(cmd)

Note: no shell=True. And you don't need to escape <> because no shell is run.

"Exec format error" might indicate that your script has invalid format, run:

$ file /usr/local/bin/script

to find out what it is. Compare the architecture with the output of:

$ uname -m

I must have accidentally deleted part of my shebang in vim... I was lost until your post. Thanks. — gloomy.penguin, Apr 15 '15 at 14:59
@nbro: wrong. Look at the very first code example in my answer. You'll get OSError if there is no shebang. It is the same error as in [your question](http://stackoverflow.com/q/35216720/4279). There is no error if you run the command in the shell (no shebang means: "run as a shell script" in this case). `subprocess.check_output()` does not run the shell by default -- no subprocess functions run the shell unless you ask explicitly (if you didn't know that; ask to reopen your question). — jfs, Feb 05 '16 at 16:33
@skygeek: such shebang is appropriate only for Python scripts. — jfs, Sep 25 '20 at 17:22
Thank you for the shebang comment! I had a space before my exclamation mark in the shebag declaration and that was causing a tool I was working with to fail — Clifford Fajardo, Jul 03 '22 at 22:50

score 12 · Answer 2 · answered Nov 16 '17 at 12:14

12

I will hijack this thread to point out that this error may also happen when target of Popen is not executable. Learnt it hard way when by accident I have had override a perfectly executable binary file with zip file.

answered Nov 16 '17 at 12:14

Drachenfels

3,037
2
32
47

that is why my answer recommends `file your-executable` command (it would show that you have a zip archive). Though some zip archives may be proper executables too e.g., [python: can executable zip files include data files?](https://stackoverflow.com/q/5355694/4279) – jfs Jan 09 '18 at 20:46
1

Hijacking this answer to point out this error also happen when trying to execute x86 executable on ARM and vice versa – Daniel Braun Mar 26 '23 at 13:39

rchang · Answer 3 · 2014-12-22T18:34:23.367

2

Have you tried this?

Out = subprocess.Popen('/usr/local/bin/script hostname = actual_server_name -p LONGLIST'.split(), shell=False,stdout=subprocess.PIPE,stderr=subprocess.PIPE)

Edited per the apt comment from @J.F.Sebastian

edited Dec 22 '14 at 18:34

answered Dec 22 '14 at 17:00

rchang

5,150
1
15
25

Hi Rchang, thanks for your feedback. It does not work. My script requires space before and after "=". Does your solution provide that? – user3477108 Dec 22 '14 at 17:11
This should work okay on this command but is not a good general purpose solution because it doesn't allow you to escape embedded spaces. The `shlex.split` method may be a better choice. – tdelaney Dec 22 '14 at 17:43
[do not use shell=True and a list argument together](http://stackoverflow.com/questions/27606653/python-subprocess-popen-woes#comment43638333_27607257) – jfs Dec 22 '14 at 18:12
1

@J.F.Sebastian Yes, you're right of course - thanks for the catch. – rchang Dec 22 '14 at 18:29

score 2 · Answer 4 · answered Dec 05 '18 at 19:06

It wouldn't be wrong to mention that Pexpect does throw a similar error

#python -c "import pexpect; p=pexpect.spawn('/usr/local/ssl/bin/openssl_1.1.0f  version'); p.interact()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/pexpect.py", line 430, in __init__
    self._spawn (command, args)
  File "/usr/lib/python2.7/site-packages/pexpect.py", line 560, in _spawn
    os.execv(self.command, self.args)
OSError: [Errno 8] Exec format error

Over here, the openssl_1.1.0f file at the specified path has exec command specified in it and is running the actual openssl binary when called.

Usually, I wouldn't mention this unless I have the root cause, but this problem was not there earlier. Unable to find the similar problem, the closest explanation to make it work is the same as the one provided by @jfs above.

what worked for me is both

adding /bin/bash at the beginning of the command or file you are
facing the problem with, or
adding shebang #!/bin/sh as the first line.

for ex.

#python -c "import pexpect; p=pexpect.spawn('/bin/bash /usr/local/ssl/bin/openssl_1.1.0f  version'); p.interact()"
OpenSSL 1.1.0f  25 May 2017

score 1 · Answer 5 · answered Dec 22 '14 at 17:10

1

If you think the space before and after "=" is mandatory, try it as separate item in the list.

Out = subprocess.Popen(['/usr/local/bin/script', 'hostname', '=', 'actual server name', '-p', 'LONGLIST'],shell=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE)

answered Dec 22 '14 at 17:10

Arun

59
7

1

downvoted. [Do not use `shell=True` and a list argument](http://bugs.python.org/issue21347) (it is an error in *most* cases). – jfs Dec 22 '14 at 18:08

OSError: [Errno 8] Exec format error

5 Answers5

Linked

Related