3

When I set a ulimit -v on a powerful machine I'm working on (48 × Intel(R) Xeon(R) CPU E7-4860 v2 @ 2.60GHz, 2 TB RAM, Red Hat Enterprise Linux Server release 6.7), python fails to import numpy when I'm working in my virtualenv, as shown below. Everything seems fine when I impose no resource limitations or when I'm not working inside virtualenv.

I suppose this is a bug somewhere. Did I set up something wrongly? Where is the correct place to report this? NumPy? OpenBLAS? System administrator?

$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 8271875
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 48000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8388604
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1024
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
$ python3.5 -c "import numpy" # success
$ ulimit -v 10000000
$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 8271875
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 48000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8388604
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1024
virtual memory          (kbytes, -v) 10000000
file locks                      (-x) unlimited
$ python3.5 -c "import numpy" # success
$ . ~/venv/stable-3.5/bin/activate
$ python3.5 -c "import numpy"
OpenBLAS blas_thread_init: pthread_create: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC 1024 current, 8271875 max
[SNIP: previous two lines shown 46 times]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/users/gholl/venv/stable-3.5/lib/python3.5/site-packages/numpy/__init__.py", line 180, in <module>
    from . import add_newdocs
  File "/home/users/gholl/venv/stable-3.5/lib/python3.5/site-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/home/users/gholl/venv/stable-3.5/lib/python3.5/site-packages/numpy/lib/__init__.py", line 8, in <module>
    from .type_check import *
  File "/home/users/gholl/venv/stable-3.5/lib/python3.5/site-packages/numpy/lib/type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "/home/users/gholl/venv/stable-3.5/lib/python3.5/site-packages/numpy/core/__init__.py", line 14, in <module>
    from . import multiarray
SystemError: initialization of multiarray raised unreported exception
Segmentation fault (core dumped)
ali_m
  • 71,714
  • 23
  • 223
  • 298
gerrit
  • 24,025
  • 17
  • 97
  • 170
  • Does the version of numpy installed outside of your virtualenv link against the same shared OpenBLAS library? – ali_m Sep 27 '16 at 16:03
  • What error does it throw? – piggs_boson Nov 25 '16 at 11:26
  • @pratyaksh `SystemError: initialization of multiarray raised unreported exception` and `Segmentation fault (core dumped)`. – gerrit Jun 28 '17 at 09:43
  • Similar to [this question](https://stackoverflow.com/questions/51256738/multiple-instances-of-python-running-simultaneously-limited-to-35/51257384#51257384), it looks as though using the virtual environment somehow pushes you to your thread limit. Do you have other things running? – Richard Jul 10 '18 at 04:48
  • Possible duplicate of [Multiple instances of Python running simultaneously limited to 35](https://stackoverflow.com/questions/51256738/multiple-instances-of-python-running-simultaneously-limited-to-35) – Richard Jul 10 '18 at 05:02

0 Answers0