How do I add more python modules to my yocto/openembedded project?

Question

I wish to add more python modules to my yocto/openembedded project but I am unsure how to? I wish to add flask and its dependencies.

Answer 1

some python packages having corresponding recipes in the meta folders, like Enum class for example:

meta-openembedded/meta-python/recipes-devtools/python/python-enum34_1.1.6.bb

unfortunately lot's of useful classes aren't available, but some might be needed for the python application. get used of installing missing packages using pip already on booted platform? but what if the target product is not IP network connected? the solution is to implement a new recipe and add to the platform meta layer (at least). Example is a recipe for the module keyboard useful for intercepting keys/buttons touch events:

1. use PyPi web site to identify if the package is available:

https://pypi.org/project/keyboard/

2. download archive available on the package description page:

https://github.com/boppreh/keyboard/archive/master.zip

3. collect some useful information required to fill-out a new recipe:

   • SUMMARY - could be obtained from the package description page
   • HOMEPAGE - the project URL on github or bitbucket or sourceforge, etc
   • LICENSE - verify license type
   • LIC_FILES_CHKSUM by executing md5sum on existing LICENSE or README or PKG-INFO file located in the root of the package (preferrably)
   • SRC_URI[md5sum] - is md5sum of the archive itself. it will be used to discover and download archive on pypi server automatically with the help of supporting script inherit pypi
   • PYPI_PACKAGE_EXT - if the package is not tar.gz require to supply the correct extension

4. create missing python-keyboard_0.13.1.bb recipe:

`

SUMMARY = "Hook and simulate keyboard events on Windows and Linux"
HOMEPAGE = "https://github.com/boppreh/keyboard"
LICENSE = "BSD-3-Clause"
LIC_FILES_CHKSUM = "file://PKG-INFO;md5=9bc8ba91101e2f378a65d36f675c88b7"

SRC_URI[md5sum] = "d4b90e53bbde888e7b7a5a95fe580a30"
SRC_URI += "file://add_missing_CHANGES_md.patch"

PYPI_PACKAGE = "keyboard"

PYPI_PACKAGE_EXT = "zip"

inherit pypi
inherit setuptools

BBCLASSEXTEND = "native nativesdk"

`

5. the package has been patched by adding

SRC_URI += "file://add_missing_CHANGES_md.patch"

directive to the recipe due to missing CHANGES.md file used by setup.py script to identify package version (this step is optional). the patch itself has to be placed inside the folder next to the recipe matching recipe name but without version:

python-keyboard

Answer 2

This question is old, but currently in 2020 there is a python package called pipoe.

pipoe can generate .bb classes corresponding to python packages for you!

Usage:

$ pip3 install pipoe
$ pipoe -p requests
OR
$ pipoe -p requests --python python3

Now copy the generated .bb files to your layer and use them.

https://pypi.org/project/pipoe/

Answer 3

The OE layer index at layers.openembedded.org lists all known layers and the recipes they contain, so searching that should bring up the meta-python layer that you can add to your build and use recipes from.

Answer 4

In your Image recipe you can add a Python module by adding it to the IMAGE_INSTALL variable:

IMAGE_INSTALL += "python-numpy"

You can find possible modules for example by searching for them with wildcards:

find -name *python*numpy*bb

in the Yocto Folder brings:

./poky/meta/recipes-devtools/python/python-numpy_1.7.0.bb

Answer 5

The pipoe did not work for me either, I ended up making this bash script. Someone else might find it useful.

You will need to change this in my script below:

local my_layers_dir="my/layers/directory"

To run this script:

./pypi.sh <modulename>
#example:
./pypi.sh humanfriendly   #this should generate the bb file for the humanfriendly python module

pypi.sh:

#!/bin/bash
set -ex

function argstovars()
{
  for change in $@; do
      set -- `echo $change | tr '=' ' '`
      eval $1=$2
  done
}

function main(){
    
    local module=""
    argstovars $@
    local my_layers_dir="my/layers/directory"
    
    local url_files="https://pypi.org/project/$module/#files"
    mkdir -p /tmp/pypi
    rm -fr /tmp/pypi/*
    pushd /tmp/pypi
    
    wget $url_files
    local targz_url=$(cat index.html | grep https://files  | grep tar.gz | sed -r "s/<a href=\"(.*)\">/\1/g")
    wget $targz_url
    local targz_file=$(ls  | grep tar.gz)
    local md5=$(md5sum $targz_file)
    md5=${md5%% *}
    local sha256=$(sha256sum $targz_file)
    sha256=${sha256%% *}
    tar -xf $targz_file
    local module_with_version=$(echo "$targz_file" | sed -r "s/(.*)\.tar\.gz/\1/g")
    pushd $module_with_version
    local license_file=$(find . -name LICENSE*)
    local md5lic=$(md5sum $license_file)
    md5lic=${md5lic%% *}
    popd
    popd

    module_with_version="${module_with_version//-/_}"; echo $foo
    mkdir -p "$my_layers_dir/$module"
    pushd "$my_layers_dir/$module"
    
    echo "SUMMARY = \"This is a python module for $module\"
HOMEPAGE = \"https://pypi.org/project/$module/\"
LICENSE = \"MIT\"
LIC_FILES_CHKSUM = \"file://$license_file;md5=$md5lic\"

SRC_URI[md5sum] = \"$md5\"
SRC_URI[sha256sum] = \"$sha256\"

PYPI_PACKAGE = \"$module\"

inherit pypi setuptools3

RDEPENDS_${PN} += \" \
    python3-psutil \
\"
" > "${module_with_version}.bb"

    popd
}

time main module=$@