Trying to use pyenv to build python versions on macOS, but every install I make ends up with
❯ python
>>> import lzma
ModuleNotFoundError: No module named '_lzma'
I have installed all of the following packages:
brew install openssl readline sqlite3 xz zlib
xcode-select --install
Only caveat is that homebrew installs packages to ~/.brew.
Any input is appreciated.
None of the prior answers worked for me. The instructions in this gist worked for me.
In short: You may be missing the xz libraries, in which case you can install them with Homebrew and then reinstall/rebuild the Python version from pyenv:
$ brew install xz
$ pyenv uninstall <desired-python-version>
$ pyenv install <desired-python-version>
Note: I only had this problem with the Python installed by pyenv, but not the Mac system Python or the conda python. It might be best to use the brewed python (brew install python) unless you have a specific need for pyenv (like needing more control over the python version/updating).
Based on pyenv wiki, you should install the desired python version with --enable-framework flag. It did work for me.
PYTHON_CONFIGURE_OPTS="--enable-framework" pyenv install 3.6.7
The reason is that the module you're trying to use is implemented in C, and CPython with shared libs will be built with the aforementioned flag.
This solved it for me:
sudo apt-get install lzma
sudo apt-get install liblzma-dev
sudo apt-get install libbz2-dev
And add this files to your current environment: e.g. (see: https://github.com/pandas-dev/pandas/issues/27532#issuecomment-822272237):
sudo cp /usr/lib/python3.8/lib-dynload/_bz2.cpython-38-x86_64-linux-gnu.so /usr/local/lib/python3.8/
sudo cp /usr/lib/python3.8/lib-dynload/_lzma.cpython-38-x86_64-linux-gnu.so /usr/local/lib/python3.8/
For example, in my runtime environment:
sudo cp /usr/lib/python3.8/lib-dynload/_bz2.cpython-38-x86_64-linux-gnu.so /home/luca/.asdf/installs/python/3.8.10/lib/python3.8`
sudo cp /usr/lib/python3.8/lib-dynload/_lzma.cpython-38-x86_64-linux-gnu.so /home/luca/.asdf/installs/python/3.8.10/lib/python3.8
Google always lead me to this thread, but a good answer is in here
Briefly, you need to do:
CFLAGS="-I$(brew --prefix xz)/include" LDFLAGS="-L$(brew --prefix xz)/lib" pyenv install 3.9.1
If you are a M1 user this will work for you under rosetta.
framework build does not work for me.
Ended up figuring out. This issue only arose when moving the Homebrew directory from its default. This was not needed when Homebrew was installed normally.
I added this to my .zshrc (alternatively, your .bashrc or .bash_profile)
export LDFLAGS="-L/Users/pcosta/.brew/opt/xz/lib $LDFLAGS"
export CPPFLAGS="-I/Users/pcosta/.brew/opt/xz/include $CPPFLAGS"
export PKG_CONFIG_PATH="/Users/pcosta/.brew/opt/xz/lib/pkgconfig:$PKG_CONFIG_PATH"
Homebrew warns that you should do this for other installed packages, but not xz. Presumably, because it is not needed if Homebrew lives where it expects to.
I don't have an answer. But this symptom sounds like this Python bug ticket:
Reported in 2020. Unfortunately still unfixed as of 2022.
I experienced this issue when trying to use package csvkit on my Windows 10 running MobaXterm:
Package installs just fine but then the /bin/csv* tools don't run:
$ /bin/python3 -m pip install csvkit --quiet
✔
$ csvstat --version
Traceback (most recent call last):
File "/bin/csvstat", line 5, in <module>
from csvkit.utilities.csvstat import launch_new_instance
File "/usr/lib/python3.6/site-packages/csvkit/utilities/csvstat.py", line 12, in <module>
from csvkit.cli import CSVKitUtility, parse_column_identifiers
File "/usr/lib/python3.6/site-packages/csvkit/cli.py", line 16, in <module>
import lzma
File "/usr/lib/python3.6/lzma.py", line 27, in <module>
from _lzma import *
ModuleNotFoundError: No module named '_lzma'
✘
My workaround was to use python2. Not super happy with that.
First throw out the non-working python3 version like so:
$ /bin/python3 -m pip uninstall csvkit --quiet --yes
✔
$ hash -r
✔
$ csvstat --version
csvstat: command not found
✘
So now it's gone. Let's try again but with python2:
$ /bin/python2 -m pip list | grep -i csv
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
WARNING: You are using pip version 20.3.4; however, version 21.3.1 is available.
You should consider upgrading via the '/bin/python2 -m pip install --upgrade pip' command.
✘
$ /bin/python2 -m pip install csvkit --quiet
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
WARNING: The scripts csvclean, csvcut, csvformat, csvgrep, csvjoin, csvjson, csvlook, csvpy, csvsort, csvsql, csvstack, csvstat, in2csv and sql2csv are installed in '/usr/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: You are using pip version 20.3.4; however, version 21.3.1 is available.
You should consider upgrading via the '/bin/python2 -m pip install --upgrade pip' command.
✔
$ csvstat --version
csvstat 1.0.7
✔
Result: csvkit is installed and /bin/csvstat.exe runs without the "_lzma" error. But unfortunately on python2 and not on python3. -- This workaround is kinda good enough for me, because I just wanted the /bin/csv* utilities but maybe useless for people who actually need to run things on python3.
A lot of the answers here require brew for Mac development. If you don't have brew, such as when your development environment is managed by nix, you can do the following:
xz source from https://tukaani.org/xz/../configure --prefix=~/xz-install within the directory (you can change the prefix to be wherever you want to be). You may have to make sure the directory exists with mkdir -p beforehand.make within the directory (note: you will need xcode command line tools to compile this).make install. This will install to whatever directory you have set with --prefix.--prefix directory is set to ~/xz-install/, install your python with CFLAGS="-I~/xz-install/include" LDFLAGS="-L~/xz-install/lib" pyenv install 3.10.4 or whatever version you want.
If you installed python via asdf, you can reinstall Python like so:
asdf uninstall python
asdf install python latest
Example output of the install command:
python-build 3.10.6 /Users/u/.asdf/installs/python/3.10.6
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.10.6.tar.xz...
-> https://www.python.org/ftp/python/3.10.6/Python-3.10.6.tar.xz
Installing Python-3.10.6...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk
Installed Python-3.10.6 to /Users/u/.asdf/installs/python/3.10.6
frist
yum install xz-devel
yum install python-backports-lzma
pip install backports.lzma
second
cd /usr/local/lib/python3.8
vi lzma.py
finally modify about line 27
try:
from _lzma import *
from _lzma import _encode_filter_properties, _decode_filter_properties
except ImportError:
from backports.lzma import *
from backports.lzma import _encode_filter_properties, _decode_filter_properties
it‘s work！I had to search for hours to get this problem to be solved.
Credit: Yolo5 issues
I solved this on Debian Buster by running:
sudo apt install liblzma-dev
After that, I recompiled Python.
I'm not an expert with Python, but it appears to have worked.
>>> import lzma
File "/usr/local/lib/python3.5/lzma.py", line 27, in <module>
from _lzma import *
ModuleNotFoundError: No module named '_lzma'
Uninstall all versions of Python3. Then reinstall only python3.5.
