os.execv(sys.executable) uses wrong Python version

Question

I want to restart a script from within itself. However on Linux os.execv ignores sys.executable and falls back to the default python version, 2.7.
Here is a minimal running example :

import os
import sys

print(sys.executable, sys.version)

os.execv(sys.executable, ["python"] + sys.argv)

Here is the output for python3 main.py (python3 referring to the default python 3 exe):


> /usr/local/bin/python3.8 3.8.0 (default, Jun  1 2022, 16:03:28) [GCC 8.3.0]

> /usr/bin/python 3.8.0 (default, Jun  1 2022, 16:03:28) [GCC 8.3.0]

> ('/usr/bin/python', '2.7.16 (default, Oct 10 2019, 22:02:15) \n[GCC 8.3.0]')

> ('/usr/bin/python', '2.7.16 (default, Oct 10 2019, 22:02:15) \n[GCC 8.3.0]')

> ('/usr/bin/python', '2.7.16 (default, Oct 10 2019, 22:02:15) \n[GCC 8.3.0]')

...

I don't quite get what part needs to be changed or what goes wrong/am I doing wrong.

This seems almost identical to this question - https://stackoverflow.com/questions/31447442/difference-between-os-execl-and-os-execv-in-python — Anya Shenanigans, Jul 01 '22 at 16:14
If you can rely on Python 3.10, it provides the original, unmangled `argv` (C-style) as [`sys.orig_argv`](https://docs.python.org/3/library/sys.html#sys.orig_argv), which would let you just do `os.execv(sys.executable, sys.orig_argv)` without needing to manually reinsert the `"python"` (that may be incorrect/misleading). Obviously doesn't seem to be an option here (since you run 3.8). — ShadowRanger, Jul 01 '22 at 16:20
There is something very strange about two different invocations claiming to be `/usr/bin/python` but with different versions... — ShadowRanger, Jul 01 '22 at 16:23
I missed that it mangles the name of sys.executable on the second execution, returning a value which doesn't correspond to the actual version of python in use. — Anya Shenanigans, Jul 01 '22 at 16:24
Changing ["pyrhon"] to ["python3"] prevents the described behavior I was able to reproduce. — Claudio, Jul 01 '22 at 22:55

Claudio · Accepted Answer · 2022-07-06T00:51:56.037

Explicit specification of the Python version in execv() does not result in a fallback to the Python 2 version like the specification only of python does. See the code ( which is not executing an endless loop):

import os, sys, time
strF = None
try: 
    f = open("counter.txt", "r")
    strF = f.read()
    f.close()
except: 
    print('except')
    f = open("counter.txt", "w")
    f.write("0")
    f.close()
    strF = "0"

if strF == '3':
    print('> strF == 3')        
    sys.exit()
    print('>> strF == 3')        
else: 
    f = open("counter.txt", "w")
    f.write(str(1+int(strF)))
    f.close()        

print(sys.executable, sys.version, strF)

time.sleep(3)

os.execv(sys.executable, ["python3.9"] + sys.argv)

Which gives following output:

$ python3.9 main.py
except
/usr/local/bin/python3.9 3.9.13 (main, May 20 2022, 21:21:14) 
[GCC 5.4.1 20160904] 0
/usr/local/bin/python3.9 3.9.13 (main, May 20 2022, 21:21:14) 
[GCC 5.4.1 20160904] 1
/usr/local/bin/python3.9 3.9.13 (main, May 20 2022, 21:21:14) 
[GCC 5.4.1 20160904] 2
> strF == 3

where the output in case of not specifying the Python version is:

$ python3.9 main.py
except
/usr/local/bin/python3.9 3.9.13 (main, May 20 2022, 21:21:14) 
[GCC 5.4.1 20160904] 0
/usr/bin/python 3.9.13 (main, May 20 2022, 21:21:14) 
[GCC 5.4.1 20160904] 1
('/usr/bin/python', '2.7.12 (default, Nov 19 2016, 06:48:10) \n[GCC 5.4.0 20160609]', '2')
> strF == 3

UPDATE (1): The only right usage of execv() for restarting a script from within itself should be:

os.execv(sys.executable, [sys.executable] + sys.argv)

With the above code there is no need for explicit specification of the Python version, so it should work with any Python version as the version must not be hard-coded.

The Python documentation ( https://docs.python.org/3/library/os.html?highlight=execv#os.execv ) appears to me not sufficiently explaining what os.execv() actually does, as giving the same value twice appears a bit weird. Maybe someone else can comment on this here? What does path in os.execv(path, args) actually mean?

The Python documentation states only that:

The “v” variants [of exec()] are good when the number of parameters is variable, with the arguments being passed in a list or tuple as the args parameter. In either case, the arguments to the child process should start with the name of the command being run, but this is not enforced.

UPDATE (2): as pointed out in the comments the way how it comes that python2 is called by python3 not already at the first, but at the second restart seem to be as follows:

Though python3 is using the argv[0] 'python' value as the sys.executable in the restarted Python script run, it actually executes the in 'path' ( is it called ""path"" to increase confusion??? ) argument to execv() passed python3. Now in the restarted version of the main.py the sys.executable value is a wrong one not reflecting the Python version which runs the script what results in running python2 ( if python2 is the default python version on the system) as executable only in the second restart.

Thanks a lot! Any idea why in-explicit might not work after the second loop? — Core taxxe, Jul 05 '22 at 18:35
You specify as executable in the `execv()` command `python` . To my current knowledge the first argument for `execv()` should be the same as first element in the parameter list. If not ... as in your code, this seem to give some weird side-effects. See the updated answer for how to avoid explicit specification of the python version. — Claudio, Jul 05 '22 at 23:49
This is a wrapper for the libc execve, whose man page says: "argv is an array of argument strings passed to the new program. By convention, the first of these strings (i.e., argv[0]) should contain the filename associated with the file being executed." — Keith, Jul 05 '22 at 23:59
My guess is that Python is using the argv[0] as the sys.executable in the new Python. When that, in turn, calls execv it will run that one. Since "python" is python2 on that system, it execs that one. — Keith, Jul 06 '22 at 00:03
@Keith : yes, this is the case as shown in the outputs. So at the first restart the executed executable is python3 as it is passed as first argument to execv, but the sys.executable has then a wrong value which on second run is the first argument in execv ( called ""path"" to increase confusion??? ) and results in executing python2, right? — Claudio, Jul 06 '22 at 00:18
Yes. Not exactly wrong, that's what it was told is the command in argv[0]. Your update is the correct solution. — Keith, Jul 06 '22 at 01:52
Thanks a lot! That makes it clear! - I was especially confused about `python` leading to python2.7 since I've overwritten the alias for that thinking it would do the trick. — Core taxxe, Jul 06 '22 at 09:18

os.execv(sys.executable) uses wrong Python version

1 Answers1