How can I properly install PyCaret in AWS Glue?
Methods I tried:
--additional-python-modulesand--python-modules-installer-optionPython library patheasy_installas described in Use AWS Glue Python with NumPy and Pandas Python Packages
I am using Glue Version 2.0. I used --additional-python-modules and set to pycaret as shown in the picture.
Then I got this error log.
INFO 2021-07-05 18:12:15,107 18690 com.amazonaws.services.glue.PythonModuleInstaller [main] Collecting pycaret Downloading https://files.pythonhosted.org/packages/da/99/18f151991b0f06107af9723417c64e304ae2133587f85ea734a90136b4ae/pycaret-2.3.1-py3-none-any.whl (261kB)Collecting numpy==1.19.5 (from pycaret) Downloading https://files.pythonhosted.org/packages/b1/e1/8c4c5632adaffc18dba4e03e97458dc1cb00583811e6982fc620b9d88515/numpy-1.19.5-cp37-cp37m-manylinux1_x86_64.whl (13.4MB)Requirement already satisfied: matplotlib in /home/spark/.local/lib/python3.7/site-packages (from pycaret)Collecting pandas-profiling>=2.8.0 (from pycaret) Downloading https://files.pythonhosted.org/packages/3b/a3/34519d16e5ebe69bad30c5526deea2c3912634ced7f9b5e6e0bb9dbbd567/pandas_profiling-3.0.0-py2.py3-none-any.whl (248kB)Collecting wordcloud (from pycaret) Downloading https://files.pythonhosted.org/packages/1b/06/0516bdba2ebdc0d5bd476aa66f94666dd0ad6b9abda723fdf28e451db919/wordcloud-1.8.1-cp37-cp37m-manylinux1_x86_64.whl (366kB)Collecting lightgbm>=2.3.1 (from pycaret) Downloading https://files.pythonhosted.org/packages/18/b2/fff8370f48549ce223f929fe8cab4ee6bf285a41f86037d91312b48ed95b/lightgbm-3.2.1-py3-none-manylinux1_x86_64.whl (2.0MB)Collecting plotly>=4.4.1 (from pycaret) Downloading https://files.pythonhosted.org/packages/95/8d/ac1560f7ccc2ace85cd1e9619bbec1975b5d2d92e6c6fdbbdaa994c6ab4d/plotly-5.1.0-py2.py3-none-any.whl (20.6MB)Collecting umap-learn (from pycaret) Downloading https://files.pythonhosted.org/packages/75/69/85e7f950bb75792ad5d666d86c5f3e62eedbb942848e7e3126513af9999c/umap-learn-0.5.1.tar.gz (80kB)Collecting scikit-plot (from pycaret) Downloading https://files.pythonhosted.org/packages/7c/47/32520e259340c140a4ad27c1b97050dd3254fdc517b1d59974d47037510e/scikit_plot-0.3.7-py3-none-any.whlCollecting Boruta (from pycaret) Downloading https://files.pythonhosted.org/packages/b2/11/583f4eac99d802c79af9217e1eff56027742a69e6c866b295cce6a5a8fc2/Boruta-0.3-py3-none-any.whl (56kB)Collecting pyod (from pycaret) Downloading https://files.pythonhosted.org/packages/71/8a/faa04a753bc32aeef00b9acf8e23d0b914b03844b89dcc6062b28e7ab1c5/pyod-0.9.0.tar.gz (105kB)Collecting yellowbrick>=1.0.1 (from pycaret) Downloading https://files.pythonhosted.org/packages/3a/15/58feb940b6a2f52d3335cccf9e5d00704ec5ba62782da83f7e2abeca5e4b/yellowbrick-1.3.post1-py3-none-any.whl (271kB)Collecting cufflinks>=0.17.0 (from pycaret) Downloading https://files.pythonhosted.org/packages/1a/18/4d32edaaf31ba4af9745dac676c4a28c48d3fc539000c29e855bd8db3b86/cufflinks-0.17.3.tar.gz (81kB)Collecting spacy<2.4.0 (from pycaret) Downloading https://files.pythonhosted.org/packages/79/1c/7c5f7541eb883181b564a8c8ba15d21b2d7b8a38ae32f31763575cf8857d/spacy-2.3.7.tar.gz (5.8MB)
Complete output from command python setup.py egg_info:
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-mrzlr566/blis/
Traceback (most recent call last):
File "/home/spark/.local/lib/python3.7/site-packages/setuptools/installer.py", line 128, in fetch_build_egg
subprocess.check_call(cmd)
File "/usr/lib64/python3.7/subprocess.py", line 363, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError:
Command '['/usr/bin/python3', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmp__iwgkr5', '--quiet', 'blis<0.8.0,>=0.4.0']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-mafqizyu/spacy/setup.py", line 252, in <module> setup_package()
File "/tmp/pip-build-mafqizyu/spacy/setup.py", line 247, in setup_package cmdclass={"build_ext": build_ext_subclass},
File "/home/spark/.local/lib/python3.7/site-packages/setuptools/__init__.py", line 143, in setup _install_setup_requires(attrs)
File "/home/spark/.local/lib/python3.7/site-packages/setuptools/__init__.py", line 138, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/home/spark/.local/lib/python3.7/site-packages/setuptools/dist.py", line 721, in fetch_build_eggs
replace_conflicting=True,
File "/home/spark/.local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 783, in resolve
replace_conflicting=replace_conflicting
File "/home/spark/.local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1066, in best_match
return self.obtain(req, installer)
File "/home/spark/.local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1078, in obtain
return installer(requirement)
File "/home/spark/.local/lib/python3.7/site-packages/setuptools/dist.py", line 777, in fetch_build_egg
return fetch_build_egg(self, req)
File "/home/spark/.local/lib/python3.7/site-packages/setuptools/installer.py", line 130, in fetch_build_egg
raise DistutilsError(str(e))
distutils.errors.DistutilsError: Command '['/usr/bin/python3', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmp__iwgkr5', '--quiet', 'blis<0.8.0,>=0.4.0']'
returned non-zero exit status 1. ----------------------------------------
INFO 2021-07-05 18:12:15,108 18691 com.amazonaws.services.glue.PythonModuleInstaller [main]
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-mafqizyu/spacy/
I tried to exclude spacy from the list of dependencies by downloading PyCaret's source code, removing spacy from requirements.txt, packaging the source code into a whl file, and trying to install PyCaret through the whl file. Then I got an error message saying, Failed building wheel for numba Failed building wheel for llvmlite Failed building wheel
Log:
[truncated because of the limit of number of characters]copying numba/_hashtable.h -> build/lib.linux-x86_64-3.7/numba copying numba/_typeof.h -> build/lib.linux-x86_64-3.7/numba copying numba/_devicearray.h -> build/lib.linux-x86_64-3.7/numba copying numba/_numba_common.h -> build/lib.linux-x86_64-3.7/numba copying numba/typed/py.typed -> build/lib.linux-x86_64-3.7/numba/typed copying numba/misc/cmdlang.gdb -> build/lib.linux-x86_64-3.7/numba/misc copying numba/pycc/modulemixin.c -> build/lib.linux-x86_64-3.7/numba/pycc copying numba/cext/dictobject.c -> build/lib.linux-x86_64-3.7/numba/cext copying numba/cext/listobject.c -> build/lib.linux-x86_64-3.7/numba/cext copying numba/cext/utils.c -> build/lib.linux-x86_64-3.7/numba/cext copying numba/cext/listobject.h -> build/lib.linux-x86_64-3.7/numba/cext copying numba/cext/cext.h -> build/lib.linux-x86_64-3.7/numba/cext copying numba/cext/dictobject.h -> build/lib.linux-x86_64-3.7/numba/cext copying numba/core/runtime/_nrt_pythonmod.c -> build/lib.linux-x86_64-3.7/numba/core/runtime copying numba/core/runtime/nrt.c -> build/lib.linux-x86_64-3.7/numba/core/runtime copying numba/core/runtime/_nrt_python.c -> build/lib.linux-x86_64-3.7/numba/core/runtime copying numba/core/runtime/nrt.h -> build/lib.linux-x86_64-3.7/numba/core/runtime copying numba/core/runtime/nrt_external.h -> build/lib.linux-x86_64-3.7/numba/core/runtime copying numba/core/annotations/template.html -> build/lib.linux-x86_64-3.7/numba/core/annotations copying numba/cuda/tests/cudadrv/data/jitlink.ptx -> build/lib.linux-x86_64-3.7/numba/cuda/tests/cudadrv/data running build_ext building 'numba._dynfunc' extension Warning: Can't read registry to find the necessary compiler setting Make sure that Python modules winreg, win32api or win32con are installed. C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC creating build/temp.linux-x86_64-3.7 creating build/temp.linux-x86_64-3.7/numba compile options: '-I/usr/include/python3.7m -c' gcc: numba/_dynfuncmo
d.c error: Command "gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.7m -c numba/_dynfuncmod.c -o build/temp.linux-x86_64-3.7/numba/_dynfuncmod.o" failed with exit status 127 ---------------------------------------- Running setup.py clean for numba Running setup.py bdist_wheel for future: started Running setup.py bdist_wheel for future: finished with status 'done' Stored in directory: /home/spark/.cache/pip/wheels/8b/99/a0/81daf51dcd359a9377b110a8a886b3895921802d2fc1b2397e Running setup.py bdist_wheel for sklearn: started Running setup.py bdist_wheel for sklearn: finished with status 'done' Stored in directory: /home/spark/.cache/pip/wheels/76/03/bb/589d421d27431bcd2c6da284d5f2286c8e3b2ea3cf1594c074 Running setup.py bdist_wheel for pynndescent: started Running setup.py bdist_wheel for pynndescent: finished with status 'done' Stored in directory: /home/spark/.cache/pip/wheels/ba/52/4e/4c28d04d144a28f89e2575fb63628df6e6d49b56c5ddd0c74e Running setup.py bdist_wheel for htmlmin: started Running setup.py bdist_wheel for htmlmin: finished with status 'done' Stored in directory: /home/spark/.cache/pip/wheels/43/07/ac/7c5a9d708d65247ac1f94066cf1db075540b85716c30255459 Running setup.py bdist_wheel for phik: started Running setup.py bdist_wheel for phik: finished with status 'done' Stored in directory: /home/spark/.cache/pip/wheels/c0/a3/b0/f27b1cfe32ea131a3715169132ff6d85653789e80e966c3bf6 Running setup.py bdist_wheel for prometheus-flask-exporter: started Running setup.py bdist_wheel for prometheus-flask-exporter: finished with status 'done' Stored in directory: /home/spark/.cache/pip/wheels/c0/e2/9c/4f3ee23964802940f81a8b476d0b9be6fb6348cb12df2e2226 Running setup.py bdist_wheel for alembic: started Running setup.py bdist_wheel for alembic: finished with status 'done' Stored in directory: /home/spark/.cache/pip/wheels/84/07/f7/12f7370ca47a66030c2edeedcc23dec26ea0ac22dcb4c4a0f3 Running setup.py bdist_wheel for databricks-cli: started Running setup.py bdist_wheel for databricks-cli: finished with status 'done' Stored in directory: /home/spark/.cache/pip/wheels/5b/24/f3/34d8e3964dac4ba849d844273c49a679111b00d5799ebb934a Running setup.py bdist_wheel for llvmlite: started Running setup.py bdist_wheel for llvmlite: finished with status 'error' Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ws60mqho/llvmlite/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpoy9cphk5pip-wheel- --python-tag cp37: running bdist_wheel /usr/bin/python3 /tmp/pip-build-ws60mqho/llvmlite/ffi/build.py LLVM version... Traceback (most recent call last): File "/tmp/pip-build-ws60mqho/llvmlite/ffi/build.py", line 220, in <module> main() File "/tmp/pip-build-ws60mqho/llvmlite/ffi/build.py", line 210, in main main_posix('linux', '.so') File "/tmp/pip-build-ws60mqho/llvmlite/ffi/build.py", line 134, in main_posix raise RuntimeError(msg) from None RuntimeError: Could not find a `llvm-config` binary. There are a number of reasons this could occur, please see: https://llvmlite.readthedocs.io/en/latest/admin-guide/install.html#using-pip for help. error: command '/usr/bin/python3' failed with exit status 1 ---------------------------------------- Running setup.py clean for llvmlite Running setup.py bdist_wheel for bottleneck: started Running setup.py bdist_wheel for bottleneck: finished with status 'error' Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ws60mqho/bottleneck/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpfy0tfce1pip-wheel- --python-tag cp37: running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-3.7 creating build/lib.linux-x86_64-3.7/bottleneck copying bottleneck/__init__.py -> build/lib.linux-x86_64-3.7/bottleneck copying bottleneck/_pytesttester.py -> build/lib.linux-x86_64-3.7/bottleneck copying bottleneck/_version.py -> build/lib.linux-x86_64-3.7/bottleneck creating build/lib.linux-x86_64-3.7/bottleneck/tests copying bottleneck/tests/__init__.py -> build/lib.linux-x86_64-3.7/bottleneck/tests copying bottleneck/tests/util.py -> build/lib.linux-x86_64-3.7/bottleneck/tests copying bottleneck/tests/input_modification_test.py -> build/lib.linux-x86_64-3.7/bottleneck/tests copying bottleneck/tests/list_input_test.py -> build/lib.linux-x86_64-3.7/bottleneck/tests copying bottleneck/tests/move_test.py -> build/lib.linux-x86_64-3.7/bottleneck/tests copying bottleneck/tests/nonreduce_test.py -> build/lib.linux-x86_64-3.7/bottleneck/tests copying bottleneck/tests/reduce_test.py -> build/lib.linux-x86_64-3.7/bottleneck/tests copying bottleneck/tests/scalar_input_test.py -> build/lib.linux-x86_64-3.7/bottleneck/tests copying bottleneck/tests/memory_test.py -> build/lib.linux-x86_64-3.7/bottleneck/tests copying bottleneck/tests/nonreduce_axis_test.py -> build/lib.linux-x86_64-3.7/bottleneck/tests creating build/lib.linux-x86_64-3.7/bottleneck/src copying bottleneck/src/bn_config.py -> build/lib.linux-x86_64-3.7/bottleneck/src copying bottleneck/src/__init__.py -> build/lib.linux-x86_64-3.7/bottleneck/src copying bottleneck/src/bn_template.py -> build/lib.linux-x86_64-3.7/bottleneck/src creating build/lib.linux-x86_64-3.7/bottleneck/benchmark copying bottleneck/benchmark/bench.py -> build/lib.linux-x86_64-3.7/bottleneck/benchmark copying bottleneck/benchmark/autotimeit.py -> build/lib.linux-x86_64-3.7/bottleneck/benchmark copying bottleneck/benchmark/__init__.py -> build/lib.linux-x86_64-3.7/bottleneck/benchmark copying bottleneck/benchmark/bench_detailed.py -> build/lib.linux-x86_64-3.7/bottleneck/benchmark creating build/lib.linux-x86_64-3.7/bottleneck/slow copying bottleneck/slow/nonreduce.py -> build/lib.linux-x86_64-3.7/bottleneck/slow copying bottleneck/slow/move.py -> build/lib.linux-x86_64-3.7/bottleneck/slow copying bottleneck/slow/__init__.py -> build/lib.linux-x86_64-3.7/bottleneck/slow copying bottleneck/slow/reduce.py -> build/lib.linux-x86_64-3.7/bottleneck/slow copying bottleneck/slow/nonreduce_axis.py -> build/lib.linux-x86_64-3.7/bottleneck/slow UPDATING build/lib.linux-x86_64-3.7/bottleneck/_version.py set build/lib.linux-x86_64-3.7/bottleneck/_version.py to '1.3.2' running build_ext running config compiling '_configtest.c': #pragma GCC diagnostic error "-Wattributes" int __attribute__((optimize("O3"))) have_attribute_optimize_opt_3(void*); int main(void) { return 0; } gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o unable to execute 'gcc': No such file or directory failure. removing: _configtest.c _configtest.o compiling '_configtest.c': #ifndef __cplusplus static inline int static_func (void) { return 0; } inline int nostatic_func (void) { return 0; } #endif int main(void) { int r1 = static_func(); int r2 = nostatic_func(); return r1 + r2; } gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o unable to execute 'gcc': No such file or directory failure. removing: _configtest.c _configtest.o compiling '_configtest.c': #ifndef __cplusplus static __inline__ int static_func (void) { return 0; } __inline__ int nostatic_func (void) { return 0; } #endif int main(void) { int r1 = static_func(); int r2 = nostatic_func(); return r1 + r2; } gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o unable to execute 'gcc': No such file or directory failure. removing: _configtest.c _configtest.o compiling '_configtest.c': #ifndef __cplusplus static __inline int static_func (void) { return 0; } __inline int nostatic_func (void) { return 0; } #endif int main(void) { int r1 = static_func(); int r2 = nostatic_func(); return r1 + r2; } gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o unable to execute 'gcc': No such file or directory failure. removing: _configtest.c _configtest.o building 'bottleneck.reduce' extension creating build/temp.linux-x86_64-3.7 creating build/temp.linux-x86_64-3.7/bottleneck creating build/temp.linux-x86_64-3.7/bottleneck/src gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/home/spark/.local/lib/python3.7/site-packages/numpy/core/include -I/usr/include/python3.7m -Ibottleneck/src -c bottleneck/src/reduce.c -o build/temp.linux-x86_64-3.7/bottleneck/src/reduce.o -O2 unable to execute 'gcc': No such file or directory error: command 'gcc' failed with exit status 1 ---------------------------------------- Running setup.py clean for bottleneck Running setup.py bdist_wheel for pandocfilters: started Running setup.py bdist_wheel for pandocfilters: finished with status 'done' Stored in directory: /home/spark/.cache/pip/wheels/93/9a/79/b2c3567908fd6209e4674ca23d9fcf005aae5fe89148913727Successfully built pyod pyLDAvis cufflinks umap-learn future sklearn pynndescent htmlmin phik prometheus-flask-exporter alembic databricks-cli pandocfiltersFailed to build numba llvmlite bottleneckInstalling collected packages: tenacity, plotly, numpy, threadpoolctl, scikit-learn, mlxtend, jupyterlab-widgets, webencodings, packaging, bleach, mistune, ipython-genutils, traitlets, pygments, jupyter-core, testpath, entrypoints, pyrsistent, zipp, typing-extensions, importlib-metadata, attrs, jsonschema, nbformat, nest-asyncio, async-generator, tornado, pyzmq, jupyter-client, nbclient, pandocfilters, defusedxml, jupyterlab-pygments, MarkupSafe, jinja2, nbconvert, ptyprocess, terminado, pickleshare, backcall, pexpect, matplotlib-inline, parso, jedi, wcwidth, prompt-toolkit, decorator, IPython, debugpy, ipykernel, prometheus-client, Send2Trash, pycparser, cffi, argon2-cffi, notebook, widgetsnbextension, ipywidgets, llvmlite, numba, pyod, lightgbm, scikit-plot, smart-open, gensim, numexpr, future, funcy, sklearn, pyLDAvis, colorlover, cufflinks, yellowbrick, Boruta, pynndescent, umap-learn, textblob, pillow, wordcloud, seaborn, requests, htmlmin, phik, pydantic, networkx, bottleneck, tangled-up-in-unicode, multimethod, PyWavelets, imagehash, visions, missingno, pandas-profiling, kmodes, imbalanced-learn, querystring-parser, greenlet, sqlalchemy, cloudpickle, gunicorn, smmap, gitdb, gitpython, protobuf, Werkzeug, itsdangerous, Flask, prometheus-flask-exporter, Mako, python-editor, alembic, sqlparse, websocket-client, docker, tabulate, databricks-cli, mlflow, pycaret Found existing installation: numpy 1.18.1 Uninstalling numpy-1.18.1: Successfully uninstalled numpy-1.18.1 Found existing installation: scikit-learn 0.22.1 Uninstalling scikit-learn-0.22.1: Successfully uninstalled scikit-learn-0.22.1 Running setup.py install for llvmlite: started Running setup.py install for llvmlite: finished with status 'error' Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ws60mqho/llvmlite/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-r7xtmu3s-record/install-record.txt --single-version-externally-managed --compile --user --prefix=: running install running build got version from file /tmp/pip-build-ws60mqho/llvmlite/llvmlite/_version.py {'version': '0.36.0', 'full': 'e6bb8d137d922bec8beeb01a237254778759becd'} running build_ext /usr/bin/python3 /tmp/pip-build-ws60mqho/llvmlite/ffi/build.py LLVM version... Traceback (most recent call last): File "/tmp/pip-build-ws60mqho/llvmlite/ffi/build.py", line 220, in <module> main() File "/tmp/pip-build-ws60mqho/llvmlite/ffi/build.py", line 210, in main main_posix('linux', '.so') File "/tmp/pip-build-ws60mqho/llvmlite/ffi/build.py", line 134, in main_posix raise RuntimeError(msg) from None RuntimeError: Could not find a `llvm-config` binary. There are a number of reasons this could occur, please see: https://llvmlite.readthedocs.io/en/latest/admin-guide/install.html#using-pip for help. error: command '/usr/bin/python3' failed with exit status 1 ----------------------------------------
INFO 2021-07-05 17:36:34,742 81650 com.amazonaws.services.glue.PythonModuleInstaller [main] Failed building wheel for numba Failed building wheel for llvmlite Failed building wheel for bottleneckCommand "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ws60mqho/llvmlite/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-r7xtmu3s-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-build-ws60mqho/llvmlite/
I attempted to install PyCaret by setting Python library path as shown in the picture below. It didn't work well because installing a Python module through Python library path didn't install dependencies automatically. I tried to give the path to PyCaret whl file along with its dependency files. It kept asking me to provide whl files that were not listed in PyCaret's requirements.txt file. So I stopped trying.
Resources I already checked:
- https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html
- https://aws.amazon.com/blogs/big-data/building-python-modules-from-a-wheel-for-spark-etl-workloads-using-aws-glue-2-0/
- Python Cannot install module spaCy
- Use AWS Glue Python with NumPy and Pandas Python Packages
- and more...
I spent many hours now. I don't know how to resolve my issue. Any suggestions or help will be greatly appreciated.
