ModuleNotFoundError: No module named 'pandas._libs.interval'

Question

Suddenly, I can't import pandas in python. I am using anaconda as package manager, but it seems that no matter how many times I uninstall and install pandas, I still get the same error:

(base) C:\>conda install pandas
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: C:\Users\au207178\Anaconda3

  added / updated specs:
    - pandas


The following NEW packages will be INSTALLED:

  blas               pkgs/main/win-64::blas-1.0-mkl
  bottleneck         pkgs/main/win-64::bottleneck-1.3.2-py38h2a96729_1
  intel-openmp       pkgs/main/win-64::intel-openmp-2021.4.0-haa95532_3556
  mkl                pkgs/main/win-64::mkl-2021.4.0-haa95532_640
  mkl-service        pkgs/main/win-64::mkl-service-2.4.0-py38h2bbff1b_0
  mkl_fft            pkgs/main/win-64::mkl_fft-1.3.1-py38h277e83a_0
  mkl_random         pkgs/main/win-64::mkl_random-1.2.2-py38hf11a4ad_0
  numexpr            pkgs/main/win-64::numexpr-2.8.1-py38hb80d3ca_0
  numpy              pkgs/main/win-64::numpy-1.21.2-py38hfca59bb_0
  numpy-base         pkgs/main/win-64::numpy-base-1.21.2-py38h0829f74_0
  pandas             pkgs/main/win-64::pandas-1.3.5-py38h6214cd6_0


Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

(base) C:\>python
Python 3.8.12 (default, Oct 12 2021, 03:01:40) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\au207178\AppData\Roaming\Python\Python310\site-packages\pandas\__init__.py", line 22, in <module>
    from pandas.compat import (
  File "C:\Users\au207178\AppData\Roaming\Python\Python310\site-packages\pandas\compat\__init__.py", line 15, in <module>
    from pandas.compat.numpy import (
  File "C:\Users\au207178\AppData\Roaming\Python\Python310\site-packages\pandas\compat\numpy\__init__.py", line 7, in <module>
    from pandas.util.version import Version
  File "C:\Users\au207178\AppData\Roaming\Python\Python310\site-packages\pandas\util\__init__.py", line 1, in <module>
    from pandas.util._decorators import (  # noqa
  File "C:\Users\au207178\AppData\Roaming\Python\Python310\site-packages\pandas\util\_decorators.py", line 14, in <module>
    from pandas._libs.properties import cache_readonly  # noqa
  File "C:\Users\au207178\AppData\Roaming\Python\Python310\site-packages\pandas\_libs\__init__.py", line 13, in <module>
    from pandas._libs.interval import Interval
ModuleNotFoundError: No module named 'pandas._libs.interval'

I have also tried running 'conda update conda'.

What's the next thing to do?

Update:

I checked, and the file is definitely there, in "C:\Users\au207178\AppData\Roaming\Python\Python310\site-packages\pandas_libs\interval.pyx".

However, I notice that part of the path to this library says 'python310'. However, my python version is 3.8.8. Is it possible that python is somehow looking in the wrong place when trying to import pandas._libs.interval? I have tried both python 3.10 (couldn't even install pandas) and python 3.9.7 (no effect).

I have also tried conda install --revision 0, which also had no effect...

Answer 1

Yes, it appears to be loading pandas from a user-level installation. User-level installs can leak into Conda environments and lead to unpredictable behavior, such as what you are seeing.

There are two routes of action of which I know. You may want to try the second one first, which would confirm the cause. However, the first option is likely more manageable going forward, since once it's done, the issue should be resolved.

Option 1: Remove External Python(s)

If you would like Conda to simply work as expected, then uninstall the user-level Python. Note that the one detected (Python 3.10) may not be the only one, so you may have to track down multiple copies. I'm not on Windows, so I can't suggest concrete steps for uninstallation.

However, you may be already using this user-level Python for other projects, so this option may not be practicable.

Option 2: Launch Python with Isolation Flags

There are some pertinent flags that Python provides that has it ignore the various sources that can lead to leaking in other site-packages. Here are the three important ones:

$ python --help
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
...
-E     : ignore PYTHON* environment variables (such as PYTHONPATH)
-s     : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S     : don't imply 'import site' on initialization
...

Try them individually, or all together, and see if this fixes the issue. Individually, you may be able to narrow down whether it is due to PYTHONPATH (-E alone would fix) or generic user-level searching (-s alone would fix).

---

Additional Thoughts

I find it odd that a Python 3.10 installation is leaking into a Python 3.8 interpreter run. Usually, the site module only loads within matching major+minor versions (e.g., 3.8 should only pick up another 3.8). This might be indicative that something else is going on. Perhaps the PYTHONPATH environment is set? (It really never should be.)

Generally, I don't recommend using the base environment for work. This is never documented in Anaconda documentation, but it is the painful truth that many long-term Conda users come to, usually after their base either breaks or get's so overloaded with packages that it takes unreasonably long for them to upgrade the conda package. Instead, create a new environment and have that activate by default instead of base.