1

I have scoured all of google's available links to address this issue, yet it still remains unsolved for me...

I get the following error when trying to import graph_tool in python3. I have both python 2.7 and 3.6 and anaconda installed. I install packages using either pip, conda, or brew. I have MacOS Sierra 10.12.3.

$ python3
Python 3.5.2 |Anaconda custom (x86_64)| (default, Jul  2 2016, 17:52:12) 
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from graph_tool.all import *
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/numpy/core/__init__.py", line 16, in <module>
    from . import multiarray
ImportError: cannot import name 'multiarray'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/graph_tool/__init__.py", line 104, in <module>
    import numpy
  File "/usr/local/lib/python3.6/site-packages/numpy/__init__.py", line 142, in <module>
    from . import add_newdocs
  File "/usr/local/lib/python3.6/site-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/usr/local/lib/python3.6/site-packages/numpy/lib/__init__.py", line 8, in <module>
    from .type_check import *
  File "/usr/local/lib/python3.6/site-packages/numpy/lib/type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "/usr/local/lib/python3.6/site-packages/numpy/core/__init__.py", line 26, in <module>
    raise ImportError(msg)
ImportError: 
Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` (removes all
files not under version control).  Otherwise reinstall numpy.

Original error was: cannot import name 'multiarray'


Some info on my directories / paths:

>>> import sys
>>> print(('\n').join(sys.path))

/usr/local/lib/python3.6/site-packages
/Users/mymacbook/anaconda/lib/python35.zip
/Users/mymacbook/anaconda/lib/python3.5
/Users/mymacbook/anaconda/lib/python3.5/plat-darwin
/Users/mymacbook/anaconda/lib/python3.5/lib-dynload
/Users/mymacbook/.local/lib/python3.5/site-packages
/Users/mymacbook/anaconda/lib/python3.5/site-packages
/Users/mymacbook/anaconda/lib/python3.5/site-packages/Sphinx-1.4.6-py3.5.egg
/Users/mymacbook/anaconda/lib/python3.5/site-packages/aeosa
/Users/mymacbook/anaconda/lib/python3.5/site-packages/setuptools-27.2.0-py3.5.egg
/usr/local/lib/python2.7/site-packages
/usr/local/Cellar/matplotlib/2.0.2/libexec/lib/python2.7/site-packages
/usr/local/Cellar/numpy/1.13.3/libexec/nose/lib/python2.7/site-packages

$ echo $PYTHONPATH
/usr/local/lib/python3.6/site-packages

$ pr /Users/mymacbook/.local/lib/python3.5/site-packages/homebrew.pth
import site; site.addsitedir("/usr/local/lib/python2.7/site-packages")


Additional information:

Somewhere online also specified that I should have the most recent version of matplotlib. I run brew info matplotlib and the following Caveat appears. I run the commands as specified but this Caveat still reappears. I'm not sure if this issue is relevant to the multiarray issue.

==> Caveats
If you want to use the `wxagg` backend, do `brew install wxpython`.
This can be done even after the matplotlib install.
If you use system python (that comes - depending on the OS X version -
with older versions of numpy, scipy and matplotlib), you may need to
ensure that the brewed packages come earlier in Python's sys.path with:
  mkdir -p /Users/mymacbook/.local/lib/python3.5/site-packages
  echo 'import sys; sys.path.insert(1, "/usr/local/lib/python2.7/site-packages")' >> /Users/mymacbook/.local/lib/python3.5/site-packages/homebrew.pth

This formula installed .pth files to Homebrew's site-packages and your
Python isn't configured to process them, so you will not be able to
import the modules this formula installed. If you plan to develop
with these modules, please run:
  mkdir -p /Users/mymacbook/.local/lib/python3.5/site-packages
  echo 'import site; site.addsitedir("/usr/local/lib/python2.7/site-packages")' >> /Users/mymacbook/.local/lib/python3.5/site-packages/homebrew.pth


FYI info on my numpy package. It also has the same Caveat as above.

$ brew info numpy
numpy: stable 1.13.3 (bottled), HEAD
Package for scientific computing with Python
http://www.numpy.org
/usr/local/Cellar/numpy/1.12.1 (442 files, 9.6MB)
  Poured from bottle on 2017-06-08 at 15:05:49
/usr/local/Cellar/numpy/1.13.0 (1,371 files, 29.0MB)
  Built from source on 2017-06-24 at 00:28:31 with: --with-python3
/usr/local/Cellar/numpy/1.13.3 (893 files, 21.8MB) *
  Poured from bottle on 2017-10-22 at 22:06:55


Also another issue (potentially related) ....

$ pip search numpy
Traceback (most recent call last):
  File "/Users/tamtran/anaconda/bin/pip", line 7, in <module>
    from pip import main
  File "/Users/tamtran/anaconda/lib/python3.5/site-packages/pip/__init__.py", line 26, in <module>
    from pip.utils import get_installed_distributions, get_prog.
.
.
.
AttributeError: '_NamespacePath' object has no attribute 'sort'

Thanks in advance for your help! (sorry for the plethora of information, just trying to provide as much context as possible) (also if you could provide specific commands to your solutions that would be appreciated)

tamtam
  • 641
  • 9
  • 24

2 Answers2

1

In my case, this problem was caused by sys.path looking in the 2.7 site-packages directory before looking in 3.6.

In your sys.path, it looks like you're having a similar issue. Calling python3 loads Python 3.5 from Anaconda. But all of the Anaconda 3.5 site-package directories are checked after the 2.7 directories.

# from OP sys.path
/usr/local/lib/python2.7/site-packages
/usr/local/Cellar/matplotlib/2.0.2/libexec/lib/python2.7/site-packages
/usr/local/Cellar/numpy/1.13.3/libexec/nose/lib/python2.7/site-packages
/Users/mymacbook/anaconda/lib/python3.5/site-packages # <-- should be before 2.7

^ That's what's causing the problem.

I tried a number of different solutions to fix this, including getting rid of .pth files and re-routing sys.path in-script. Eventually I just removed the numpy directory from the 2.7 site-packages, as I didn't need it, and I was frustrated trying to trace exactly how sys.path was constructed and re-order it permanently.

If you don't want to be that extreme, have a look at the site docs if you want to trace exactly how sys.path is put together.

andrew_reece
  • 20,390
  • 3
  • 33
  • 58
  • Hi andrew_reece, thanks for your answer! In my sys.path as noted above, python3.6/site-packages already precedes python2.7/site-packages, therefore sys.path is not the source of the issue - right? – tamtam Oct 23 '17 at 00:56
  • I don't think so - with `python3`, you got Anaconda 3.5, according to your output: `Python 3.5.2 |Anaconda custom (x86_64)`. I think that means you need to get to the 3.5 Anaconda `site-package` before `sys.path` hits 2.7. – andrew_reece Oct 23 '17 at 00:57
  • Ok I have moved python2.7/site-packages to the bottom position of sys.path. I have also manually deleted the numpy folder from /usr/local/lib/python2.7/site-packages, but the problem still remains... – tamtam Oct 23 '17 at 01:15
  • Are `/usr/local/Cellar/matplotlib/2.0.2/libexec/lib/python2.7/site-packages` and `/usr/local/Cellar/numpy/1.13.3/libexec/nose/lib/python2.7/site-packages` still present and ahead of the Anaconda 3.5 listings? – andrew_reece Oct 23 '17 at 01:19
  • Oops - just fixed that by moving them to the end. Problem still persists unfortunately.. – tamtam Oct 23 '17 at 01:35
  • Sorry to hear it, `sys.path` bug tracking is painful. Are you still getting exactly the same Traceback? I.e., first line is `"/usr/local/lib/python2.7/site-packages/graph_tool/__init__.py", line 104`? Also, do you want to be on 3.5 or 3.6, and in Anaconda or not? It might be worth actually going through the `site` docs I linked to and setting `sys.path` build correctly. – andrew_reece Oct 23 '17 at 01:43
  • Yes I am getting exactly the same traceback. I want to be on either 3.5 or 3.6, but it seems that my system has 3.6 and Anaconda has 3.5... and I'm not sure about keeping Anaconda.. I just want to be able to access jupyter notebooks (Idk the relationship). I clicked your link and couldn't derive any useful information :/ – tamtam Oct 23 '17 at 11:18
1

Attempts to reshuffle the sys.path here will not lead to happiness.

You cannot mix-and-match brew, conda, and pip packages from python 2 and 3 and have reproducible results

Create yourself a fresh conda environment

conda create -n tensorproject python ipython tensorflow <other pkgs>
source activate tensorflowproject

conda will manage your dependecies for you. You can conda install additional packages and find non-anaconda ones on conda-forge. When you need more, you can pip install into that environment.

Note, that when you create a conda environment like that, only packages known to work together will be placed in the site packages and site.py will have no strangeness.

Note: you should also probably drop the PYTHONPATH environment variable.

Edit: graph_tool installation From the graph_tool installation page:

Make sure you use the same compiler to compile the whole stack (Python, Boost, etc) or you may lead into problems.

The compiler requirements for graph_tool (below) don't match your Anaconda build (py35).

Phil Cooper
  • 5,747
  • 1
  • 25
  • 41
  • I agree that reshuffling `sys.path` isn't optimal. But I hit OP's same issue after starting with a fresh `conda` environment - `sys.path` was still not operating correctly. – andrew_reece Oct 23 '17 at 14:57
  • @andrew_reece (see my edit) if you or OP post a specific target Anaconda version & platform, I or somone else might be able to take a run at building the package for upload to Anaconda or conda-forge. – Phil Cooper Oct 23 '17 at 17:57
  • @PhilCooper Hi Phil, thanks for your reply! I have attempted to follow your suggestion, but the need for `python3.6` to install `graph_tool` has arisen, which has led me onto another journey of difficulties. Would you mind taking a look at my [new SO question here] (https://stackoverflow.com/questions/46900020/python-conda-resolvepackagenotfound)? Thanks in advance! – tamtam Oct 23 '17 at 23:41