Invoking python3 with no arguments results in the interpreter opening a script called dis.py in the current directory. How to avoid similar problems?

Question

Invoking the python 3.10.6 interpreter with no arguments produces the following output in the presence of a (possibly empty) file called dis.py in the working directory.

Python 3.10.6 (main, Aug 30 2022, 04:58:14) [Clang 13.1.6 (clang-1316.0.21.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Failed calling sys.__interactivehook__
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site.py", line 447, in register_readline
    import rlcompleter
  File "/opt/homebrew/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/rlcompleter.py", line 34, in <module>
    import inspect
  File "/opt/homebrew/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/inspect.py", line 59, in <module>
    for k, v in dis.COMPILER_FLAG_NAMES.items():
AttributeError: module 'dis' has no attribute 'COMPILER_FLAG_NAMES'
>>>

Clearly I shouldn't have my own files called dis.py lying around in the working directory! Are the names of all of the other modules included by inspect.py also "reserved"? Shouldn't the modules inspect.py imports be somehow fully qualified to stop this from happening? Any suggestions on what best practice might be would be helpful. This is a surprising behaviour and I am not aware of any warnings about it.

The `dis` module is fully qualified, it's a top-level module in the Python standard library. Generally, it's a bad idea to use the same names as standard library modules or packages in your own code, at least a the top level. Put your own stuff in a package, if you want more safety over its names (you should then only need to worry about the top-level package name). — Blckknght, Sep 23 '22 at 22:33
Just to clarify, I meant the fully qualified file name. IE the file from which the module was loaded would be hardwired. But it looks like there's no good way to arrange that (https://stackoverflow.com/a/67692/14768587) — Theo H, Sep 23 '22 at 22:42

score 1 · Accepted Answer · answered Sep 24 '22 at 10:11

I was equally surprised and alarmed after reading this question. I often run python with no arguments just to do some arithmetic or call help(), and I've never stopped to worry about what directory I'm in and whether I trust the code in it.

I did some digging and found that it's because of the order of initialization of the interpreter: A bunch of stuff is imported, then an empty string (meaning the current working directory) is prepended to sys.path, then site.py is imported. And for interactive shells, site.py imports readline and rlcompleter, which imports inspect, which imports many things, like dis and ast. If the current working directory contains readline.py or rlcompleter.py or ast.py or dis.py etc. it will be executed. In Python 3.8 and earlier rlcompleter did not import inspect, and so the number of problematic names was smaller (just readline and rlcompleter as far as I can tell).

This has been addressed in Python 3.11.0 beta 1, in two ways:

gh-92345: pymain_run_python() now imports readline and rlcompleter before sys.path is extended to include the current working directory of an interactive interpreter.

gh-57684: Add the -P command line option and the PYTHONSAFEPATH environment variable to not prepend a potentially unsafe path to sys.path.

Note that -P and PYTHONSAFEPATH address a broader issue. Even non-interactive python interpreters normally prepend something to sys.path, either the current working directory or the directory containing the main script, and that can have malicious or accidental effects. Disabling that sys.path modification avoids those pitfalls but is backward-incompatible with some existing code. Still, I hope -P can become the default behavior someday.

What can you do today, with Python 3.10 (or earlier)? One idea would be to set your PYTHONSTARTUP environment variable (which affects only interactive interpreters) to point to a script that removes any empty string from the front of sys.path, like so:

import sys
if sys.path and not sys.path[0]:
    sys.path.pop(0)

On occasions where you need to interactively import things from the current directory, you can either start the python shell with PYTHONSTARTUP= python3 or interactively enter sys.path.insert(0,'').

Invoking python3 with no arguments results in the interpreter opening a script called dis.py in the current directory. How to avoid similar problems?

1 Answers1