$PATH determines the search behavior of the shell, when the shell is asked to run a command that does not have an absolute path name. That is:
python
means "find some command named python that can be run". A shell that uses $PATH (bash, sh, zsh, and many others) will iterate through each colon-separated component element in $PATH and try them in the order they appear.
(Side note: csh and tcsh use $path, lowercase, which is an array built at shell start-up time from $PATH. You can set $PATH before running them, or you can, e.g., set path = ($path /usr/local/bin) to add another path to the end of the $path array; setting $path exports a new $PATH setting. Not many people still use tcsh as it's quite klunky and modern bash or zsh does everything tcsh does, but in the old days when computers were mostly measured in "kilo" or "mega" instead of "giga" units, tcsh was a lot smaller and faster.)
In order to run something, though, the "something"—whatever it is—must actually be executable. That is, the exec* family of system calls and functions (execve being the most basic one) will ask the OS to replace the current process with a new one constructed by running some command. The system call taks a path name (/usr/local/bin/python for instance) and tries to open that file "for execution". The file must have the executable bit set (chmod +x) and have a valid "magic number" for the OS.
On macOS (and Linux and BSDs and Unix-derived systems in general), one special valid magic number is the one formed by the characters #!. If the file begins with this string, the OS will read in the first block or so of the file—the details here are extremely OS-specific as they usually depend on the underlying virtual-memory and/or block-I/O system—and extract the first line, perhaps up to some maximum length (in the old days about 32 or 64 bytes, these days likely much longer). This line should be terminated by a newline only (CRLF here messes things up on a lot of systems). The line should consist of the #!, an optional blank, the path name of an interpreter, and then an optional blank and optional argument to pass to the program.
The exact details of handling here depend on the OS: some allow more than one argument, separated by spaces, some only one. Some allow multiple blanks after #!, some only one. In any case, the #! line—called a shebang line, as ElpieKay commented and as noted in this Wikipedia entry—determines which interpreter is to be used. The OS shifts the argv arguments up as needed to place the shebang line argument(s) in front of the remaining argv arguments passed to execve, so that if the script starts with:
#! /usr/bin/env python
then execve("/path/to/script", argv, environ); acts almost as if the program in question had invoked, instead:
argv = insert_at(1, "python", argv);
(in C using a hypothetical insert_at, or argv.insert(1, "python") in Python) followed by:
execve("/usr/bin/env", argv, environ);
So if a Python script begins with #! /usr/bin/env python, and is made executable, it will be run by the /usr/bin/env "interpreter" with argument python.
What /usr/bin/env does in this case is replicate the shell's search with $PATH (!). So this is a sort of poor-man's version of:
#! /bin/sh
exec python $@
except that it takes no extra line in the file, which means that the entire file can be a Python script. This avoids the need to use polyglot programming techniques.
The whole point of all of this is to avoid hardcoding the path to the Python interpreter. If we know that python is actually /usr/local/bin/python3 we can just write:
#! /usr/local/bin/python3
and invoke the correct interpreter. But on some systems, python is python2, or python is in /usr/bin, or python is somewhere in /opt (as yours is), or whatever. We can't really solve the whole python2-vs-python3 problem here, but we can easily handle the path-searching issue by indirecting through /usr/bin/env, which is always in /usr/bin.
Why does git assume it needs to use the command line tools (XCode) version of Python, even though it is not set anywhere in the path?
Git makes no such assumption: git commit tries to execve the .git/hooks/pre-commit file. (Git does not need to do $PATH searching here as it knows precisely where this file lives within the repository proper, in the hooks directory. In other places, Git will do the $PATH searching, the same way the shell does, sometimes by using the shell, and sometimes using a library routine that does the same thing.)
That file should be executable and start with #!. Look at it and see what it contains: it might be a shell script, or it might be a script in some other language. If it is a shell script, it might contain a lot of code, perhaps including code that sets and exports its PATH variable. That might cause it to find the XCode version of Python if and when it tries to invoke python without a path.
Now, as to:
/Applications/Xcode.app/Contents/Developer/usr/bin/python3:
No module named pre_commit
Remember that Python reads $PYTHONPATH from the environment to set up Python's sys.path. Import directives then look in sys.path, using a rather fancy mechanism so that Python can import "eggs" and "wheels" as well as actual files (see What is a Python egg? and its answers and links, and note that PEP-427 discourages directly importing wheels).
For these reasons, a lot of Python things use a bit of shell scripting first. For instance, Python "virtual environments" work by manipulating both PATH and PYTHONPATH before invoking a specific Python version.
