4

I have CentOS 7 and I have installed mpicc (it works and compiles for openmpi in C). I also have python 2.7.5 and just installed pip. I'm running this command and get the following errors:

sudo pip install mpi4py

Collecting mpi4py   Using cached mpi4py-2.0.0.tar.gz Installing
> collected packages: mpi4py   Running setup.py install for mpi4py ...
> error
>     Complete output from command /usr/bin/python2 -u -c "import setuptools,
> tokenize;__file__='/tmp/pip-build-x5jD4O/mpi4py/setup.py';exec(compile(getattr(tokenize,
> 'open', open)(__file__).read().replace('\r\n', '\n'), __file__,
> 'exec'))" install --record /tmp/pip-mpMoZO-record/install-record.txt
> --single-version-externally-managed --compile:
>     running install
>     running build
>     running build_src
>     running build_py
>     creating build
>     creating build/lib.linux-x86_64-2.7
>     creating build/lib.linux-x86_64-2.7/mpi4py
>     copying src/__main__.py -> build/lib.linux-x86_64-2.7/mpi4py
>     copying src/__init__.py -> build/lib.linux-x86_64-2.7/mpi4py
>     creating build/lib.linux-x86_64-2.7/mpi4py/include
>     creating build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/mpi4py.MPI.h -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/mpi4py.MPI_api.h -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/mpi4py.h -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/__init__.pxd -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/libmpi.pxd -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/MPI.pxd -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/__init__.pyx -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/mpi.pxi -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/mpi4py.i -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/MPI.pxd -> build/lib.linux-x86_64-2.7/mpi4py
>     copying src/libmpi.pxd -> build/lib.linux-x86_64-2.7/mpi4py
>     running build_clib
>     MPI configuration: [mpi] from 'mpi.cfg'
>     checking for library 'lmpe' ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
>     gcc -pthread _configtest.o -llmpe -o _configtest
>     /bin/ld: cannot find -llmpe
>     collect2: error: ld returned 1 exit status
>     failure.
>     removing: _configtest.c _configtest.o
>     building 'mpe' dylib library
>     creating build/temp.linux-x86_64-2.7
>     creating build/temp.linux-x86_64-2.7/src
>     creating build/temp.linux-x86_64-2.7/src/lib-pmpi
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-2.7/src/lib-pmpi/mpe.o
>     creating build/lib.linux-x86_64-2.7/mpi4py/lib-pmpi
>     gcc -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-2.7/src/lib-pmpi/mpe.o -o
> build/lib.linux-x86_64-2.7/mpi4py/lib-pmpi/libmpe.so
>     checking for library 'vt-mpi' ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
>     gcc -pthread _configtest.o -lvt-mpi -o _configtest
>     /bin/ld: cannot find -lvt-mpi
>     collect2: error: ld returned 1 exit status
>     failure.
>     removing: _configtest.c _configtest.o
>     checking for library 'vt.mpi' ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
>     gcc -pthread _configtest.o -lvt.mpi -o _configtest
>     /bin/ld: cannot find -lvt.mpi
>     collect2: error: ld returned 1 exit status
>     failure.
>     removing: _configtest.c _configtest.o
>     building 'vt' dylib library
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c src/lib-pmpi/vt.c -o build/temp.linux-x86_64-2.7/src/lib-pmpi/vt.o
>     gcc -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-2.7/src/lib-pmpi/vt.o -o
> build/lib.linux-x86_64-2.7/mpi4py/lib-pmpi/libvt.so
>     checking for library 'vt-mpi' ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
>     gcc -pthread _configtest.o -lvt-mpi -o _configtest
>     /bin/ld: cannot find -lvt-mpi
>     collect2: error: ld returned 1 exit status
>     failure.
>     removing: _configtest.c _configtest.o
>     checking for library 'vt.mpi' ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
>     gcc -pthread _configtest.o -lvt.mpi -o _configtest
>     /bin/ld: cannot find -lvt.mpi
>     collect2: error: ld returned 1 exit status
>     failure.
>     removing: _configtest.c _configtest.o
>     building 'vt-mpi' dylib library
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c src/lib-pmpi/vt-mpi.c -o build/temp.linux-x86_64-2.7/src/lib-pmpi/vt-mpi.o
>     gcc -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-2.7/src/lib-pmpi/vt-mpi.o -o
> build/lib.linux-x86_64-2.7/mpi4py/lib-pmpi/libvt-mpi.so
>     checking for library 'vt-hyb' ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
>     gcc -pthread _configtest.o -lvt-hyb -o _configtest
>     /bin/ld: cannot find -lvt-hyb
>     collect2: error: ld returned 1 exit status
>     failure.
>     removing: _configtest.c _configtest.o
>     checking for library 'vt.ompi' ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
>     gcc -pthread _configtest.o -lvt.ompi -o _configtest
>     /bin/ld: cannot find -lvt.ompi
>     collect2: error: ld returned 1 exit status
>     failure.
>     removing: _configtest.c _configtest.o
>     building 'vt-hyb' dylib library
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c src/lib-pmpi/vt-hyb.c -o build/temp.linux-x86_64-2.7/src/lib-pmpi/vt-hyb.o
>     gcc -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-2.7/src/lib-pmpi/vt-hyb.o -o
> build/lib.linux-x86_64-2.7/mpi4py/lib-pmpi/libvt-hyb.so
>     running build_ext
>     MPI configuration: [mpi] from 'mpi.cfg'
>     checking for MPI compile and link ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c _configtest.c -o _configtest.o
>     _configtest.c:2:17: fatal error: mpi.h: No such file or directory
>      #include <mpi.h>
>                      ^
>     compilation terminated.
>     failure.
>     removing: _configtest.c _configtest.o
>     error: Cannot compile MPI programs. Check your configuration!!!

I tried every solution I found so far and none seemed to work, has anyone any idea about this problem, please? Thank you

Andriy Ivaneyko
  • 20,639
  • 6
  • 60
  • 82
Logofatu114
  • 51
  • 1
  • 2
  • You say you have installed OpenMPI. But is it in your path? I mean, can you call from a new shell mpicc or trying to autocomplete mpirun command? Plus, I assume you have the OpenMPI development headers installed... – fedepad Jan 03 '17 at 15:21
  • when i run the command mpirun -np 4 program.o for a c program, it works, and when i run mpirun -np 16 python program.py for a python program, it says that i'm missing the mpi4py module – Logofatu114 Jan 03 '17 at 15:24
  • An answer has been posted, that's where I was pointing with my comment when I said "I assume you have the OpenMPI development headers installed". As said in the answer, check those and install them with the recipe given there! – fedepad Jan 03 '17 at 15:27
  • Looking at https://bitbucket.org/mpi4py/mpi4py/issues/5/help-desired-to-install-mpi4py , have you tried to load the mpi module prior to the install of mpi4py? Could you try something like `module avail` to list the available modules and something similar to `module load mpi/gcc/openmpi-1.6.4` as shown in http://chtc.cs.wisc.edu/MPIuseguide ? – francis Jan 03 '17 at 19:37

4 Answers4

14

Run into the same issue and solved with:

yum install openmpi-devel
export CC=/usr/lib64/openmpi/bin/mpicc
pip install mpi4py
Iakov Nakhimovski
  • 847
  • 7
  • 5
  • On newer Centos System that path might change. Check with `which mpicc`. If that does not return anything, run `mpi-selector-menu` to set a systemwide default and check again. – renefritze Apr 28 '20 at 08:56
  • just for append: I solved it by update PATH: `export PATH=/usr/lib64/openmpi/bin:$PATH` – Kevin Chan Aug 07 '23 at 08:33
0

it seems you missed to install the development files of OpenMPI on Centos, the line that is the key here is:

_configtest.c:2:17: fatal error: mpi.h: No such file or directory
  #include <mpi.h>

you should install the openmpi-devel (or equivalent) through yum and you should be good to reinstall the mpi4py module.

Rafael Aguilar
  • 3,084
  • 1
  • 25
  • 31
  • 1
    I already did that, sudo yum install openmpi-devel, this was the first thing I did after I installed CentOS. The OpenMPI works for programs written in C, i can write, compile and execute programs written in C using openmpi. – Logofatu114 Jan 03 '17 at 15:30
  • It seems that `mpi4py` uses another dependencies `yum install mpich-devel mpich-autoload`. Source: https://bitbucket.org/mpi4py/mpi4py/issues/5/help-desired-to-install-mpi4py – Rafael Aguilar Jan 03 '17 at 15:55
  • Something very weird is happening, so as i said, i installed openmpi, i tested it with mpirun, it worked, and after i try to install the mpi4py, when i type mpirun it says command not found – Logofatu114 Jan 03 '17 at 17:42
  • and same happens on root, wtf is this? – Logofatu114 Jan 03 '17 at 17:42
  • i installed those mpich packages and it still happens. – Zuoanqh Jan 02 '18 at 13:24
  • @Logofatu114 but now compilation is OK?, just to have more details about problem – Rafael Aguilar Jan 02 '18 at 14:39
0

You could also try:

CentOS

yum install python-mpi4py
yum install python3-mpi4py

Debian

apt install python-mpi4py
apt install python3-mpi4py
BostonGeorge
  • 3,092
  • 1
  • 17
  • 12
0

For anyone else who runs into this, I'm posting this answer just in case you have a similar problem to me. The answer from @francis was the key. On my Centos7.7 system I was able to install mpi4py using the CC environment variable suggested by @iakov-nakhimovski but it was not working and I was getting runtime errors. The problem turned out to be that other software installed on the system had its own python installation in the path which was interfering with the setup of mpi4py. The solution was to remove this other installation from the path and to load the openmpi module which was shipped with the openmpi-devel package (I think).

module load mpi/openmpi-x86_64

Then install mpi4py using pip (note you do not need to explicitly define CC=mpicc since all these tools get put into the path by the environment module). When running a python script which requires mpi4py you also need to load this module.

user3708611
  • 1