Python - what is the shebang line supposed to do if the stated version is not installed ?? what if shebang is not the first line?

Question

I'm having to update a legacy Python 2 program and as such I figured I'd use the shebang line to confirm the correct version of Python is used in certain contexts. I haven't used the shebang line before and apparently I'm unclear on it's purpose and action.

Currently I'm using a computer with a relatively fresh Ubuntu 18.04.5 install and Python 2 is not installed yet (Python 3 is installed). I made a test script named test.py with the following contents:

#!/usr/bin/env python2
import sys
print(sys.version)

and ran it as follows and received the following results:

$ python3 test.py 
3.6.9 (default, Oct  8 2020, 12:12:24) 
[GCC 8.4.0]

As you can see the program ran and printed the version, which I don't understand. Is this really the expected behavior? Since Python 2 is not installed I would have expected a result along the lines of:

ERROR: Python 2 is not installed

Also I have a 2nd question, but before I ask that I should mention I did the following:

$ chmod +x test.py
$ ./test.py

and received the following result:

/usr/bin/env: ‘python2’: No such file or directory

Which would be my expected results given the script and these commands, so at least my faith in the shebang line is partially restored, which brings me to my second question. If I modify test.py so the shebang is no longer the first line as follows:

# test.py
#!/usr/bin/env python2
import sys
print(sys.version)

And then run it:

$ ./test.py

The terminal hangs for about 30 seconds, then says this:

import-im6.q16: not authorized `sys' @ error/constitute.c/WriteImage/1037.
./test.py: line 4: syntax error near unexpected token `sys.version'
./test.py: line 4: `print(sys.version)'

Which is exactly the same result I get as if I remove the shebang line entirely. Is this the expected behavior, i.e. if the shebang line is not the first line but is otherwise correct is it supposed to do nothing?

Answer 1

The shebang is what will be run if the script is run with chmod +x test.py and then ./test.py. Then the shell (i.e. bash) will run /usr/bin/env python2 test.py. However, when you explicitly use python3 test.py the shell will not check your shebang, it will just run python3.

Also, if you put the shebang at any line other than the first, it is no longer a shebang. Quoting Wikipedia:

In computing, a shebang is the character sequence consisting of the characters number sign and exclamation mark (#!) at the beginning of a script.

The shsebang can only be at the biginning. If it is not then the shell will not look at it, so it ran test.py as a shell script, where many commands are not found.

Answer 2

When you do python3 test.py, you're already telling your system to execute the file with python3. The shebang is serving no purpose here. Shebang is for when you need to specify the executing command inside the file. So if you're adding a shebang, you're assuming those who use your code will execute it with ./test after changing the file's mode to executable.

Answer 3

So the main differences between the python3 test.py and ./test.py is how the system is trying to run the script. the first attempt python3 test.py is running the script directly through an installed python interpreter and will anything that is valid python. As you know the # in python indicates all that follows is a comment and should be ignored, so as far as the interpreter is concerned that first shebang doesn't "exist".

Now running scripts from the shell is a tad different. When running a script directly, the shebang is used to tell your system how it is to run the script / what interpreter to use. Therefore when you tried './test.py' your system tried to find the specified interpreter python2 and couldn't find it. So if your shebang was #!/usr/bin/env bash you computer would try to run your script as a bash script and you'll probably be thrown a lot of syntax errors.

The shebang is not necessary for a python script but makes it simpler to use in many cases, after all its more convenient to type ./script.sh than bash script.sh. The end user does not necessarily need to know how your script is run, only how to run it.