Can't find `lz4` header when installing `roslz4` with pip on a Mac

Question

I'm trying to install roslz4 on my Mac, and I get the following error.

      In file included from ros_comm/utilities/roslz4/src/_roslz4module.c:37:
      include/roslz4/lz4s.h:38:10: fatal error: 'lz4.h' file not found
      #include <lz4.h>
               ^~~~~~~
      1 error generated.
      error: command 'clang' failed with exit status 1
      [end of output]

To counter this, I installed lz4 with brew which ran successfully

> brew install lz4
==> Downloading https://ghcr.io/v2/homebrew/core/lz4/manifests/1.9.3
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/lz4/blobs/sha256:d222923849a6fefd391ab6705f1468c10d287c08ab9c4b5053a18a552139e262
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:d222923849a6fefd391ab6705f1468c10d287c08ab9c4b5053a18a552139e262?se=2022-06-14T22%3A20%3A00Z&s
######################################################################## 100.0%
==> Pouring lz4--1.9.3.arm64_monterey.bottle.tar.gz
  /opt/homebrew/Cellar/lz4/1.9.3: 22 files, 620.6KB
==> Running `brew cleanup lz4`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).

However, it does not appear to have resolved the problem.

Full pip output:

> pip install --extra-index-url https://rospypi.github.io/simple/ roslz4                
Looking in indexes: https://pypi.org/simple, https://rospypi.github.io/simple/
Collecting roslz4
  Using cached https://github.com/rospypi/simple/raw/any/roslz4/roslz4-1.14.3.post2.tar.gz (18 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: roslz4
  Building wheel for roslz4 (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [21 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-10.14-arm64-3.8
      creating build/lib.macosx-10.14-arm64-3.8/roslz4
      copying ros_comm/utilities/roslz4/src/roslz4/__init__.py -> build/lib.macosx-10.14-arm64-3.8/roslz4
      running build_ext
      building '_roslz4' extension
      creating build/temp.macosx-10.14-arm64-3.8
      creating build/temp.macosx-10.14-arm64-3.8/ros_comm
      creating build/temp.macosx-10.14-arm64-3.8/ros_comm/utilities
      creating build/temp.macosx-10.14-arm64-3.8/ros_comm/utilities/roslz4
      creating build/temp.macosx-10.14-arm64-3.8/ros_comm/utilities/roslz4/src
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -DXXH_NAMESPACE=ROSLZ4_ -Iinclude -Iroscpp_core/cpp_common/include -I/Users/cmauceri/Workspace/run-mgmt/jupyter_venv/include -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c ros_comm/utilities/roslz4/src/_roslz4module.c -o build/temp.macosx-10.14-arm64-3.8/ros_comm/utilities/roslz4/src/_roslz4module.o -Wno-strict-prototypes -Wno-missing-field-initializers -Wno-unused-variable -Wno-strict-aliasing
      In file included from ros_comm/utilities/roslz4/src/_roslz4module.c:37:
      include/roslz4/lz4s.h:38:10: fatal error: 'lz4.h' file not found
      #include <lz4.h>
               ^~~~~~~
      1 error generated.
      error: command 'clang' failed with exit status 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for roslz4
  Running setup.py clean for roslz4
Failed to build roslz4
Installing collected packages: roslz4
  Running setup.py install for roslz4 ... error
  error: subprocess-exited-with-error
  
  × Running setup.py install for roslz4 did not run successfully.
  │ exit code: 1
  ╰─> [21 lines of output]
      running install
      running build
      running build_py
      creating build
      creating build/lib.macosx-10.14-arm64-3.8
      creating build/lib.macosx-10.14-arm64-3.8/roslz4
      copying ros_comm/utilities/roslz4/src/roslz4/__init__.py -> build/lib.macosx-10.14-arm64-3.8/roslz4
      running build_ext
      building '_roslz4' extension
      creating build/temp.macosx-10.14-arm64-3.8
      creating build/temp.macosx-10.14-arm64-3.8/ros_comm
      creating build/temp.macosx-10.14-arm64-3.8/ros_comm/utilities
      creating build/temp.macosx-10.14-arm64-3.8/ros_comm/utilities/roslz4
      creating build/temp.macosx-10.14-arm64-3.8/ros_comm/utilities/roslz4/src
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -DXXH_NAMESPACE=ROSLZ4_ -Iinclude -Iroscpp_core/cpp_common/include -I/Users/cmauceri/Workspace/run-mgmt/jupyter_venv/include -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c ros_comm/utilities/roslz4/src/_roslz4module.c -o build/temp.macosx-10.14-arm64-3.8/ros_comm/utilities/roslz4/src/_roslz4module.o -Wno-strict-prototypes -Wno-missing-field-initializers -Wno-unused-variable -Wno-strict-aliasing
      In file included from ros_comm/utilities/roslz4/src/_roslz4module.c:37:
      include/roslz4/lz4s.h:38:10: fatal error: 'lz4.h' file not found
      #include <lz4.h>
               ^~~~~~~
      1 error generated.
      error: command 'clang' failed with exit status 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> roslz4

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Do I need to add the location of lz4 headers to an environment variable? How can I get pip to find the header?

score 1 · Accepted Answer · answered Jun 16 '22 at 09:53

After doing brew install lz4 the following (from this SO post by rgov) worked for me without the need of any source code changes:

CFLAGS="-I$(brew --prefix lz4)/include" \
LDFLAGS="-L$(brew --prefix lz4)/lib" \
pip install roslz4 --extra-index-url https://rospypi.github.io/simple/

score 0 · Answer 2 · answered Jun 14 '22 at 22:43

Answer from a coworker:

My solution was to download the roslz4 tarball and manually add the include folders and library symbols into the setup config, then run python setup.py install

Here are the first few lines of my setup.py file (which includes my addition to include and lib)

import platform
from setuptools import setup, Extension

extra_compile_args = []
include_dirs = [
    "include",
    "roscpp_core/cpp_common/include",
    "/usr/local/Cellar/lz4/1.9.3/include",
]
libraries = ["lz4"]
library_dirs = ["/usr/local/Cellar/lz4/1.9.3/lib"]
define_macros = [
    ("XXH_NAMESPACE", "ROSLZ4_"),
]

My changes are the lines with "Cellar" and "library_dirs"

Can't find `lz4` header when installing `roslz4` with pip on a Mac

2 Answers2