0

I want to use pymc. Additionally, I am required to use poetry. Hence, conda is not really an option. The installation (poetry add pymc yields the same result for me as pip install) works fine. Unfortunately, I get a CompileError. The heavy lifting is done via pytensor, which gives the error eventually.

The minimal code to generate the error is the one given on the homepage:

import numpy
import pytensor.tensor as at
from pytensor import function
x = at.dscalar('x')
y = at.dscalar('y')
z = x + y
f = function([x, y], z)

The error is the following, indicating a missing Python header file:

CompileError: Compilation failed (return status=1):
/usr/bin/g++ -shared -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -march=skylake -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes -mno-sha -mpclmul -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -msgx -mbmi2 -mno-pconfig -mno-wbnoinvd -mno-tbm -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mrdrnd -mf16c -mfsgsbase -mrdseed -mprfchw -madx -mfxsr -mxsave -mxsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mclflushopt -mxsavec -mxsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-avx5124fmaps -mno-avx5124vnniw -mno-clwb -mno-mwaitx -mno-clzero -mno-pku -mno-rdpid -mno-gfni -mno-shstk -mno-avx512vbmi2 -mno-avx512vnni -mno-vaes -mno-vpclmulqdq -mno-avx512bitalg -mno-avx512vpopcntdq -mno-movdiri -mno-movdir64b -mno-waitpkg -mno-cldemote -mno-ptwrite --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=skylake -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -I/clv/.venv/lib/python3.10/site-packages/numpy/core/include -I/usr/include/python3.10 -I/clv/.venv/lib/python3.10/site-packages/pytensor/link/c/c_code -L/usr/lib/x86_64-linux-gnu -L/usr/lib -fvisibility=hidden -o /home/.pytensor/compiledir_Linux-5.15--generic-x86_64-with-glibc2.31-x86_64-3.10.11-64/tmpux_8vf6f/mace4c18ebddf6a38476ff0675d34e7f736090323052aaf44e25e4dcefc693ebf.so /home/.pytensor/compiledir_Linux-5.15--generic-x86_64-with-glibc2.31-x86_64-3.10.11-64/tmpux_8vf6f/mod.cpp -lpython3.10
/home/.pytensor/compiledir_Linux-5.15--generic-x86_64-with-glibc2.31-x86_64-3.10.11-64/tmpux_8vf6f/mod.cpp:1:10: fatal error: Python.h: No such file or directory
    1 | #include <Python.h>
      |          ^~~~~~~~~~
compilation terminated.

Apply node that caused the error: Add(x, y)
Toposort index: 0
Inputs types: [TensorType(float64, shape=()), TensorType(float64, shape=())]

Backtrace when the node is created (use PyTensor flag traceback__limit=N to make it longer):
  File "/clv/.venv/lib/python3.10/site-packages/IPython/terminal/interactiveshell.py", line 881, in interact
    self.run_cell(code, store_history=True)
  File "/clv/.venv/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3009, in run_cell
    result = self._run_cell(
  File "/clv/.venv/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3064, in _run_cell
    result = runner(coro)
  File "/clv/.venv/lib/python3.10/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner
    coro.send(None)
  File "/clv/.venv/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3269, in run_cell_async
    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
  File "/clv/.venv/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3448, in run_ast_nodes
    if await self.run_code(code, result, async_=asy):
  File "/clv/.venv/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3508, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-1-7f7bbd6c6521>", line 11, in <module>
    z = x + y

HINT: Use a linker other than the C linker to print the inputs' shapes and strides.
HINT: Use the PyTensor flag `exception_verbosity=high` for a debug print-out and storage map footprint of this Apply node.

I realized that the compiler directory gets created on the flight.

Since there is not much I find about this issue, I almost suspect that anyone else gets it running via pip. I also used to have this problem within a docker contain, which now works. But since plotting is difficult within a contain, it does not fit my workflow.

I would be very happy, if someone has an idea how to fix that.

itmatters
  • 665
  • 6
  • 8
  • What OS is this on? you need the development headers for Python. – M S Aug 18 '23 at 12:02
  • It is on ubuntu `uname -a` yields: 5.15.0-79-generic #86~20.04.2-Ubuntu SMP Mon Jul 17 23:27:17 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux. How do I get the headers manually? – itmatters Aug 21 '23 at 07:29
  • 1
    `sudo apt-get install python-dev -y` this should put Python.h file into /usr/include/python3.10/ – M S Aug 21 '23 at 07:55
  • beautiful, thanks a lot. In my case it actually was `install python3.10-dev`. You gave me the decisive hint. It seemed to be a topic here already: https://stackoverflow.com/questions/11041299/python-h-no-such-file-or-directory – itmatters Aug 22 '23 at 07:23

0 Answers0