1

To all the experts - we have been developing an application that contains following modules:

  • opencv
  • numpy
  • pandas
  • stomp
  • nltk
  • bs4
  • PIL
  • requests

and a few others those are not part of pythons standard library. Was looking to package the same into single executable. Hence tried working out with pyinstaller. The same when executed - it dose build up the distribution folder - but it fails to execute. It throws the following error:

Traceback (most recent call last):
File "app.py", line 6, in <module>
File "/root/Projects/BIUEngine/lib/python3.6/site-packages/PyInstaller/loader/pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "ProcessManager.py", line 7, in <module>
File "/root/Projects/BIUEngine/lib/python3.6/site-packages/PyInstaller/loader/pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "ProcessWorker.py", line 16, in <module>
File "/root/Projects/BIUEngine/lib/python3.6/site-packages/PyInstaller/loader/pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "site-packages/cv2/__init__.py", line 3, in <module>
ImportError: numpy.core.multiarray failed to import
[3502] Failed to execute script app

Below is the output of the command when run

root@devserver:~/Projects/BIUEngine/src# pyinstaller app.py 
113 INFO: PyInstaller: 3.4
114 INFO: Python: 3.6.7
116 INFO: Platform: Linux-4.4.0-131-generic-x86_64-with-Ubuntu-16.04-xenial
116 INFO: wrote /root/Projects/BIUEngine/src/app.spec
119 INFO: UPX is not available.
123 INFO: Extending PYTHONPATH with paths
['/root/Projects/BIUEngine/src', '/root/Projects/BIUEngine/src']
123 INFO: checking Analysis
123 INFO: Building Analysis because Analysis-00.toc is non existent
124 INFO: Initializing module dependency graph...
127 INFO: Initializing module graph hooks...
179 INFO: Analyzing base_library.zip ...
10300 INFO: running Analysis Analysis-00.toc
10357 INFO: Caching module hooks...
10381 INFO: Analyzing /root/Projects/BIUEngine/src/app.py
13258 INFO: Processing pre-find module path hook   distutils
13297 INFO: distutils: retargeting to non-venv dir '/usr/lib/python3.6/distutils'
14908 INFO: Processing pre-safe import module hook   setuptools.extern.six.moves
16017 INFO: Processing pre-find module path hook   site
16044 INFO: site: retargeting to fake-dir '/root/Projects/BIUEngine/lib/python3.6/site-packages/PyInstaller/fake-modules'
18473 INFO: Processing pre-safe import module hook   six.moves
32950 INFO: Processing pre-safe import module hook   urllib3.packages.six.moves
55571 INFO: Loading module hooks...
55572 INFO: Loading module hook "hook-PIL.Image.py"...
56546 INFO: Loading module hook "hook-encodings.py"...
56688 INFO: Loading module hook "hook-PIL.SpiderImagePlugin.py"...
56699 INFO: Excluding import 'tkinter'
56704 INFO: Import to be excluded not found: 'FixTk'
56704 INFO: Loading module hook "hook-distutils.py"...
56753 INFO: Loading module hook "hook-certifi.py"...
56755 INFO: Loading module hook "hook-sysconfig.py"...
56792 INFO: Loading module hook "hook-pytz.py"...
56989 INFO: Loading module hook "hook-nltk.py"...
57816 INFO: Loading module hook "hook-pkg_resources.py"...
58813 INFO: Processing pre-safe import module hook   win32com
58976 INFO: Loading module hook "hook-PIL.py"...
58988 INFO: Import to be excluded not found: 'FixTk'
58988 INFO: Excluding import 'PySide'
58996 INFO:   Removing import of PySide from module PIL.ImageQt
59000 INFO: Excluding import 'PyQt5'
59007 INFO:   Removing import of PyQt5 from module PIL.ImageQt
59010 INFO: Excluding import 'PyQt4'
59018 INFO:   Removing import of PyQt4 from module PIL.ImageQt
59025 INFO: Excluding import 'tkinter'
59036 INFO:   Removing import of tkinter from module PIL.ImageTk
59038 INFO: Loading module hook "hook-xml.py"...
59040 INFO: Loading module hook "hook-pandas.py"...
60447 INFO: Loading module hook "hook-sqlite3.py"...
60607 INFO: Loading module hook "hook-numpy.py"...
60609 INFO: Loading module hook "hook-pycparser.py"...
61578 INFO: Loading module hook "hook-lib2to3.py"...
61616 INFO: Loading module hook "hook-cv2.py"...
61617 INFO: Loading module hook "hook-xml.etree.cElementTree.py"...
61619 INFO: Loading module hook "hook-setuptools.py"...
62934 INFO: Loading module hook "hook-pydoc.py"...
62936 INFO: Loading module hook "hook-numpy.core.py"...
63088 INFO: Looking for ctypes DLLs
63294 WARNING: library user32 required via ctypes not found
63399 INFO: Analyzing run-time hooks ...
63439 INFO: Including run-time hook 'pyi_rth_pkgres.py'
63458 INFO: Including run-time hook 'pyi_rth_nltk.py'
63463 INFO: Including run-time hook 'pyi_rth_multiprocessing.py'
63536 INFO: Looking for dynamic libraries
65726 INFO: Looking for eggs
65727 INFO: Python library not in binary dependencies. Doing additional searching...
65822 INFO: Using Python library /usr/lib/x86_64-linux-gnu/libpython3.6m.so.1.0
65882 INFO: Warnings written to /root/Projects/BIUEngine/src/build/app/warn-app.txt
66237 INFO: Graph cross-reference written to /root/Projects/BIUEngine/src/build/app/xref-app.html
66476 INFO: checking PYZ
66477 INFO: Building PYZ because PYZ-00.toc is non existent
66478 INFO: Building PYZ (ZlibArchive) /root/Projects/BIUEngine/src/build/app/PYZ-00.pyz
70637 INFO: Building PYZ (ZlibArchive) /root/Projects/BIUEngine/src/build/app/PYZ-00.pyz completed successfully.
70722 INFO: checking PKG
70722 INFO: Building PKG because PKG-00.toc is non existent
70722 INFO: Building PKG (CArchive) PKG-00.pkg
70837 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
70839 INFO: Bootloader /root/Projects/BIUEngine/lib/python3.6/site-packages/PyInstaller/bootloader/Linux-64bit/run
70840 INFO: checking EXE
70840 INFO: Building EXE because EXE-00.toc is non existent
70840 INFO: Building EXE from EXE-00.toc
70842 INFO: Appending archive to ELF section in EXE /root/Projects/BIUEngine/src/build/app/app
70967 INFO: Building EXE from EXE-00.toc completed successfully.
70981 INFO: checking COLLECT
70982 INFO: Building COLLECT because COLLECT-00.toc is non existent
70983 INFO: Building COLLECT COLLECT-00.toc
82471 INFO: Building COLLECT COLLECT-00.toc completed successfully.

Can anyone please help me with the same .. as how i can resolve the issue?

Blakdronzer
  • 300
  • 3
  • 13
  • Where are you trying to run the executable? Is it the same computer you have run pyinstaller on? Or is it a different machine? – heilerich Jan 24 '19 at 07:09
  • Yes - was trying to run the executable and on the same machine on which i ran pyinstaller. – Blakdronzer Jan 24 '19 at 08:12
  • It may not be a pyinstaller issue but a numpy issue. See https://stackoverflow.com/questions/20518632/importerror-numpy-core-multiarray-failed-to-import. – Tim Jan 24 '19 at 23:32

2 Answers2

1

The ImportError: numpy.core.multiarray failed to import error is infamous for causing crashes related to numpy trying to import platform specific shared libraries. Almost always it's either caused by trying to import the wrong version or by not finding those libraries at all.

Probably your packaged executable is trying to import some framework from outside the package and experiencing some kind of version mismatch. Most likely you're also working with some kind of virtual environment like conda or virtualenv?

Without having access to the machine it's very hard to pinpoint the issue. I would try the following to gather further information on the issue:

  1. Try to run your executable on another machine that has no python or at least no numpy installed at all and see what happens.

  2. Try to repack your application on a machine that has just one version of numpy installed.

  3. Carefully investigate which version of and from where numpy is actually imported by running python -c 'import numpy; print(numpy.__version__); print(numpy.__file__)' in the environment, you're creating the executable in, as well as in the executable itself.

The last step can be achieved by putting

import numpy
import numpy
print(numpy.__version__)
print(numpy.__file__)

in your app.py as the very first lines (i.e. to see the output before it's crashing)

heilerich
  • 944
  • 8
  • 14
0

Thank you all out who are helping out there around. I also had posted elsewhere - and with God's grace - someone did come up with a solution around for the answer. It worked out for me. What he suggested was - i may have 2 or more different numpy installed for various versions in the machine. One for 2.7 other for 3.6 and so on. He recommended me removing either of them and check if things worked out. It did make some sense - so i wen ahead and cleaned up all the numpy from 2.7 / 3.6 and other packages those were manually installed for the project as required. Post that - i just created a virtual machine and made a fresh installation of the required packages (along with numpy).

Also - made sure that the pyinstaller - while building did look in for numpy.core.multiarray as a hidden import. (Second step - i already had tried out in past).

What to say - things did workout for me. It stopped giving me the error it used to give in for that. Now am fixing other issues - but surely this method / mechanism worked out for me.

Thank you all for helping me out again. Love you all.

Blakdronzer
  • 300
  • 3
  • 13