Trying to add python3-simplejpeg to Yocto fails with Cmake error

Question

I tried to add simplejpeg to a Yocto project currently running on the main/bleeding edge version.

I used the latest pipoe version to autogenerate a .bb / receipe file with the simple command pipoe -p simplejpeg --python python3

this generated following bb file:

SUMMARY = "A simple package for fast JPEG encoding and decoding."
HOMEPAGE = ""
AUTHOR = "Joachim Folz <joachim.folz@dfki.de>"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=af754d278ce3e4ddb2683b219eb831ba"

inherit setuptools3

SRC_URI = "https://files.pythonhosted.org/packages/25/50/6a4ea4f14d72d21c0fe0e3ef4c916b162fc6208f7a32c145f690c7c678b0/simplejpeg-1.6.6.tar.gz"
SRC_URI[md5sum] = "8f25a777a21832be0771ee22f282ce88"
SRC_URI[sha256sum] = "942294f9fd602375289759f1b70dd734e1c939063d8479c04dffb23074a8d662"

S = "${WORKDIR}/simplejpeg-1.6.6"

DEPENDS += " "
RDEPENDS:${PN} = "python3-numpy"

BBCLASSEXTEND = "native nativesdk"

trying to build the receipe with bitbake ( bitbake python3-simplejpeg ) results in following error:

NOTE: Executing Tasks
ERROR: python3-simplejpeg-1.6.6-r0 do_compile: 'python3 setup.py bdist_wheel ' execution failed.
ERROR: python3-simplejpeg-1.6.6-r0 do_compile: ExecutionError('/home/user/yocto/build/tmp/work/cortexa53-poky-linux/python3-simplejpeg/1.6.6-r0/temp/run.do_compile.10750', 1, None, None)
ERROR: Logfile of failure stored in: /home/user/yocto/build/tmp/work/cortexa53-poky-linux/python3-simplejpeg/1.6.6-r0/temp/log.do_compile.10750
Log data follows:
| DEBUG: Executing shell function do_compile
| Traceback (most recent call last):
|   File "/home/user/yocto/build/tmp/work/cortexa53-poky-linux/python3-simplejpeg/1.6.6-r0/simplejpeg-1.6.6/setup.py", line 11, in <module>
|     from cmake import CMAKE_BIN_DIR
| ModuleNotFoundError: No module named 'cmake'
| ERROR: 'python3 setup.py bdist_wheel ' execution failed.
| WARNING: exit code 1 from a shell command.
ERROR: Task (/home/user/yocto/sources/meta-project/recipes-project/recipes-core/python3-simplejpeg/python3-simplejpeg_1.6.6.bb:do_compile) failed with exit code '1'

I also tired rewriting a lot of this receipe in hopes of getting it working...

SUMMARY = "python3-simplejpeg"

LICENSE = "CLOSED"

SRC_URI[md5sum] = "8f25a777a21832be0771ee22f282ce88"
SRC_URI[sha256sum] = "942294f9fd602375289759f1b70dd734e1c939063d8479c04dffb23074a8d662"

PYPI_PACKAGE = "simplejpeg"

DEPENDS += "cmake-native python3-setuptools-native python3-numpy-native"

inherit cmake pypi setuptools3 python3native pkgconfig

S = "${WORKDIR}/simplejpeg-1.6.6"

do_configure() {
   setuptools3_do_configure
   cmake_do_configure
}

do_compile() {
   setuptools3_do_compile
   cmake_do_compile
}

do_install() {
   setuptools3_do_install
   cmake_do_install
}

BBCLASSEXTEND = "native nativesdk"

but this just ended in the error


NOTE: Executing Tasks
ERROR: python3-simplejpeg-1.6.6-r0 do_configure: ExecutionError('/home/user/yocto/build/tmp/work/cortexa53-poky-linux/python3-simplejpeg/1.6.6-r0/temp/run.do_configure.17786', 1, None, None)
ERROR: Logfile of failure stored in: /home/user/yocto/build/tmp/work/cortexa53-poky-linux/python3-simplejpeg/1.6.6-r0/temp/log.do_configure.17786
Log data follows:
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are ['/home/user/yocto/sources/poky/meta/recipes-core/glibc/glibc_2.37.bb:do_populate_sysroot', '/home/user/yocto/sources/poky/meta/recipes-devtools/cmake/cmake-native_3.26.4.bb:do_populate_sysroot', '/home/user/yocto/sources/poky/meta/recipes-devtools/gcc/gcc-cross_13.1.bb:do_populate_sysroot', '/home/user/yocto/sources/poky/meta/recipes-devtools/gcc/gcc-runtime_13.1.bb:do_populate_sysroot', '/home/user/yocto/sources/poky/meta/recipes-devtools/python/python3_3.11.3.bb:do_populate_sysroot', '/home/user/yocto/sources/poky/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_populate_sysroot', 'virtual:native:/home/user/yocto/sources/poky/meta/recipes-devtools/ninja/ninja_1.11.1.bb:do_populate_sysroot', 'virtual:native:/home/user/yocto/sources/poky/meta/recipes-devtools/patch/patch_2.7.6.bb:do_populate_sysroot', 'virtual:native:/home/user/yocto/sources/poky/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_populate_sysroot', 'virtual:native:/home/user/yocto/sources/poky/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot', 'virtual:native:/home/user/yocto/sources/poky/meta/recipes-devtools/python/python3-build_0.10.0.bb:do_populate_sysroot', 'virtual:native:/home/user/yocto/sources/poky/meta/recipes-devtools/python/python3-installer_0.7.0.bb:do_populate_sysroot', 'virtual:native:/home/user/yocto/sources/poky/meta/recipes-devtools/python/python3-numpy_1.24.3.bb:do_populate_sysroot', 'virtual:native:/home/user/yocto/sources/poky/meta/recipes-devtools/python/python3-setuptools_67.6.1.bb:do_populate_sysroot', 'virtual:native:/home/user/yocto/sources/poky/meta/recipes-devtools/python/python3-wheel_0.40.0.bb:do_populate_sysroot', 'virtual:native:/home/user/yocto/sources/poky/meta/recipes-devtools/python/python3_3.11.3.bb:do_populate_sysroot']
| NOTE: Installed into sysroot: []
| NOTE: Skipping as already exists in sysroot: ['glibc', 'cmake-native', 'gcc-cross-aarch64', 'gcc-runtime', 'python3', 'quilt-native', 'ninja-native', 'patch-native', 'pkgconfig-native', 'pseudo-native', 'python3-build-native', 'python3-installer-native', 'python3-numpy-native', 'python3-setuptools-native', 'python3-wheel-native', 'python3-native', 'attr-native', 'libtool-native', 're2c-native', 'python3-flit-core-native', 'zlib-native', 'ncurses-native', 'zstd-native', 'openssl-native', 'bzip2-native', 'xz-native', 'libgcc', 'linux-libc-headers', 'python3-cython-native', 'opkg-utils', 'libxcrypt', 'expat', 'libtirpc', 'openssl', 'libffi', 'libedit', 'gdbm', 'util-linux-libuuid', 'bzip2', 'xz', 'libnsl2', 'zlib', 'ncurses', 'sqlite3', 'util-linux-libuuid-native', 'libedit-native', 'libffi-native', 'expat-native', 'gdbm-native', 'sqlite3-native', 'libnsl2-native', 'libtirpc-native', 'python3-pyproject-hooks-native', 'python3-packaging-native', 'flex-native', 'gnu-config-native', 'gmp-native', 'texinfo-dummy-native', 'libmpc-native', 'binutils-cross-aarch64', 'mpfr-native', 'gettext-minimal-native', 'unzip-native', 'perl-native', 'm4-native', 'make-native']
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_configure
| CMake Warning:
|   Ignoring extra path from command line:
|
|    "/home/user/yocto/build/tmp/work/cortexa53-poky-linux/python3-simplejpeg/1.6.6-r0/simplejpeg-1.6.6"
|
|
| CMake Error: The source directory "/home/user/yocto/build/tmp/work/cortexa53-poky-linux/python3-simplejpeg/1.6.6-r0/simplejpeg-1.6.6" does not appear to contain CMakeLists.txt.
| Specify --help for usage, or press the help button on the CMake GUI.
| WARNING: exit code 1 from a shell command.
ERROR: Task (/home/user/yocto/sources/meta-project/recipes-project/recipes-core/python3-simplejpeg/python3-simplejpeg_1.6.6.bb:do_configure) failed with exit code '1'
NOTE: Tasks Summary: Attempted 1346 tasks of which 1337 didn't need to be rerun and 1 failed.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 1 seconds

could someone please help me get this fixed?

Thanks a lot :)

Update 1:

SUMMARY = "A simple package for fast JPEG encoding and decoding."
HOMEPAGE = ""
AUTHOR = "Joachim Folz <joachim.folz@dfki.de>"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=af754d278ce3e4ddb2683b219eb831ba"

SRC_URI = "https://files.pythonhosted.org/packages/ac/35/b20cad2bee9d91e901f16beaa9d26ff4685dd17ad727918774b32ec04485/simplejpeg-${PV}-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl;downloadfilename=simplejpeg-${PV}-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.zip;subdir=${BP}"

SRC_URI[md5sum] = "0a29f9057f0f6c2a3b06d4df53e5f5b0"
SRC_URI[sha256sum] = "50b282cc9e03a315d452e8fb541cfe0d9b8e0dff1869a90828faab4c99d70331"

inherit python3-dir

LICENSE = "MIT"
LIC_FILES_CHKSUM = "file:///${S}/simplejpeg-${PV}.dist-info/LICENSE;md5=af754d278ce3e4ddb2683b219eb831ba"

do_unpack[depends] += "unzip-native:do_populate_sysroot"

DEPENDS += "python3"

FILES:${PN} += "\
    ${libdir}/${PYTHON_DIR}/site-packages/simplejpeg/* \
    ${libdir}/${PYTHON_DIR}/site-packages/simplejpeg-${PV}.dist-info/* \
"

do_install() {
    install -d ${D}${libdir}/${PYTHON_DIR}/site-packages/simplejpeg
    install -d ${D}${libdir}/${PYTHON_DIR}/site-packages/simplejpeg-${PV}.dist-info
    install -m 644 ${S}/simplejpeg/* ${D}${libdir}/${PYTHON_DIR}/site-packages/simplejpeg/
    install -m 644 ${S}/simplejpeg-${PV}.dist-info/* ${D}${libdir}/${PYTHON_DIR}/site-packages/simplejpeg-${PV}.dist-info/
}

I added a new .bb and tried to build it not from source, but from a .whl file. However, I get a new error and even though the packet seems to be installed, no data is available in the final image

WARNING: python3-simplejpeg-1.6.6-r0 do_populate_sysroot: File '/usr/lib/python3.11/site-packages/simplejpeg/_jpeg.cpython-311-aarch64-linux-gnu.so' from python3-simplejpeg was already stripped, this will prevent future debugging!
ERROR: python3-simplejpeg-1.6.6-r0 do_package: QA Issue: File '/usr/lib/python3.11/site-packages/simplejpeg/_jpeg.cpython-311-aarch64-linux-gnu.so' from python3-simplejpeg was already stripped, this will prevent future debugging! [already-stripped]
ERROR: python3-simplejpeg-1.6.6-r0 do_package: Fatal QA errors were found, failing task.
ERROR: Logfile of failure stored in: /home/user/yocto/build/tmp/work/cortexa53-poky-linux/python3-simplejpeg/1.6.6-r0/temp/log.do_package.2645
ERROR: Task (/home/user/yocto/sources/meta-project/recipes-project/recipes-core/python3-simplejpeg/python3-simplejpeg_1.6.6.bb:do_package) failed with exit code '1'

Update 2 - finally working thanks to @skandigraun

SUMMARY = "A simple package for fast JPEG encoding and decoding."
HOMEPAGE = ""
AUTHOR = "Joachim Folz <joachim.folz@dfki.de>"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=af754d278ce3e4ddb2683b219eb831ba"

SRC_URI = "https://files.pythonhosted.org/packages/ac/35/b20cad2bee9d91e901f16beaa9d26ff4685dd17ad727918774b32ec04485/simplejpeg-${PV}-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl;downloadfilename=simplejpeg-${PV}-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.zip;subdir=${BP}"

SRC_URI[md5sum] = "0a29f9057f0f6c2a3b06d4df53e5f5b0"
SRC_URI[sha256sum] = "50b282cc9e03a315d452e8fb541cfe0d9b8e0dff1869a90828faab4c99d70331"

inherit python3-dir

LICENSE = "MIT"
LIC_FILES_CHKSUM = "file:///${S}/simplejpeg-${PV}.dist-info/LICENSE;md5=af754d278ce3e4ddb2683b219eb831ba"

do_unpack[depends] += "unzip-native:do_populate_sysroot"

INSANE_SKIP:${PN} += "already-stripped"

DEPENDS += "python3"

FILES:${PN} += "\
    ${libdir}/${PYTHON_DIR}/site-packages/simplejpeg/* \
    ${libdir}/${PYTHON_DIR}/site-packages/simplejpeg-${PV}.dist-info/* \
"

do_install() {
    install -d ${D}${libdir}/${PYTHON_DIR}/site-packages/simplejpeg
    install -d ${D}${libdir}/${PYTHON_DIR}/site-packages/simplejpeg-${PV}.dist-info
    install -m 644 ${S}/simplejpeg/* ${D}${libdir}/${PYTHON_DIR}/site-packages/simplejpeg/
    install -m 644 ${S}/simplejpeg-${PV}.dist-info/* ${D}${libdir}/${PYTHON_DIR}/site-packages/simplejpeg-${PV}.dist-info/
}

In the first version of the recipe you should add `python3-cmake-native` to `DEPENDS` - a script is looking for python cmake module. — skandigraun, Jun 26 '23 at 04:24
Thank you, I added it to DEPENDS, got another cmake error? ````ERROR: python3-simplejpeg-1.6.6-r0 do_compile: 'python3 setup.py bdist_wheel ' execution failed. | DEBUG: Executing shell function do_compile [....] | copying simplejpeg/_jpeg.pyi -> build/lib.linux-x86_64-cpython-311/simplejpeg | running build_ext | error: [Errno 2] No such file or directory: '/home/user/yocto/build/tmp/work/cortexa53-poky-linux/python3-simplejpeg/1.6.6-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/cmake/data/bin/cmake' | ERROR: 'python3 setup.py bdist_wheel ' execution failed. ```` — nmaas87, Jun 26 '23 at 05:52
Ugh... right... apparently that setup script is somewhat opinionated when it comes to compiling. That particular error can be patched, but now I see this package wasn't created with cross compiling in mind. Let me see if I can find a not too painful way — skandigraun, Jun 26 '23 at 06:35
Thanks @skandigraun for your insights - do you have further information on how to get it working? Thanks a lot for your tmie :)! — nmaas87, Jun 29 '23 at 06:12
Yeah, sorry for that. I was looking into it... In the setup.py it directly calls cmake with native settings, that make it fail. In theory it would be possible to patch setup.py, and bake in yocto's cross compiling settings, but the more I look at it, the more my heart aches. Alternatively, what would be your target arch? Cortex53 sounds like aarch64, which has prebuilt artifacts in the pypi repo: https://pypi.org/project/simplejpeg/#files - you could go a different way, and instead of compiling it yourself, you could just install the precompiled package. — skandigraun, Jun 29 '23 at 14:33
If you go the precompiled route, just the other day there was a question about installing precompiled whl files, it supposed to work more or less universally: https://stackoverflow.com/questions/76495515 — skandigraun, Jun 29 '23 at 14:36
Thanks a lot @skandigraun - yeah, it would be for aarch64 similar targets like RPi0W2 or CM4. I was hoping to compile it from scratch to be not dependant from whl releases (especially if sth breaks and want to try a quick compile of latest/Main release), however I understand your concern. I will try and have a look if I can get this working, thanks a lot already :) — nmaas87, Jun 29 '23 at 14:42
@skandigraun - I tried to go the wheel route now and could create a bb for that (see within original post) but get an error already-stripped and can confirm that the program is not deployed to my image. Any ideas? :) — nmaas87, Jun 30 '23 at 20:11
Adding `INSANE_SKIP:${PN} += "already-stripped"` to the recipe should skip this check and let the build finish. — skandigraun, Jul 01 '23 at 05:55
@skandigraun Yes it did, but sadly no data was available in the finished image... I could not find the module in the dist-packages folder — nmaas87, Jul 01 '23 at 08:43
Are you sure? I just compiled and installed your recipe successfully. After installing python-numpy also, I could import simplejpeg successfully in python console — skandigraun, Jul 01 '23 at 14:43
Awesome, thank you very much @skandigraun it works now. I tried to do the same skip in a different version from the local.conf and that did surpress the error, however it did not install the files for some reason. Anyway, it works, thanks! :) — nmaas87, Jul 01 '23 at 19:45

Trying to add python3-simplejpeg to Yocto fails with Cmake error

0 Answers0