1

When trying to import xgboost with the following:

import pandas as pd,  numpy as np, psycopg2, xgboost as xgb, os, shutil, pickle, random

I get the following error:

  File "/opt/virtual_environments/ml_venv/lib/python3.6/site-packages/xgboost/__init__.py", line 9, in <module>
    from .core import DMatrix, DeviceQuantileDMatrix, Booster, DataIter
  File "/opt/virtual_environments/ml_venv/lib/python3.6/site-packages/xgboost/core.py", line 203, in <module>
    _LIB = _load_lib()
  File "/opt/virtual_environments/ml_venv/lib/python3.6/site-packages/xgboost/core.py", line 194, in _load_lib
    """)
xgboost.core.XGBoostError: 
XGBoost Library (libxgboost.so) could not be loaded.
Likely causes:
  * OpenMP runtime is not installed
    - vcomp140.dll or libgomp-1.dll for Windows
    - libomp.dylib for Mac OSX
    - libgomp.so for Linux and other UNIX-like OSes
    Mac OSX users: Run `brew install libomp` to install OpenMP runtime.

  * You are running 32-bit Python on a 64-bit OS

Error message(s): ['dlopen: cannot load any more object with static TLS']

Switching the import of xgboost in front of psycopg2 like so:

import pandas as pd,  numpy as np, xgboost as xgb, psycopg2, os, shutil, pickle, random

The code imports, finds the shared libraries and runs fine.

I know this could be dependent on the setup of the server/environment, but is there some reason why its working when xgboost is imported first? Is psycopg2 potentially changing system paths or environment variables?

Cheers

Lachlan Allison
  • 11
  • 1
  • 2
  • The error message is `dlopen: cannot load any more object with static TLS`. From [a similar issue here](https://github.com/scikit-learn/scikit-learn/issues/14485#issuecomment-522218682): *There is a maximum amount of such "static TLS" slots*. You might want to look into TLS and what psycopg2/xgboost do with it. – leosh Mar 25 '22 at 01:43
  • It seems to be related to the compilation of the library and you might not be able to do anything about it on your side. Other related issues: https://stackoverflow.com/questions/40270183/importerror-dlopen-cannot-load-any-more-object-with-static-tls and https://stackoverflow.com/questions/14892101/cannot-load-any-more-object-with-static-tls – leosh Mar 25 '22 at 01:49
  • 1
    Cheers @leosh, some good info in those links. I'll check with the sys admins but not a big issue if I can just reorder the imports. – Lachlan Allison Mar 30 '22 at 05:08

0 Answers0