Why does the yocto bblayers.conf file use absolute paths?

Question

The yocto project allows the use of relative path in most of its configuration files but not within the ./build/conf/bblayers.conf file. What is the reason for blocking the use of anything but absolute paths for the BBLAYERS and BBLAYERS_NON_REMOVABLE variables?

I have looked at the BitBake user manual for yocto version 2.0 (current release) but that does not explain the reasoning. I also checked some of the older manual versions but they do not seem to mention the reasoning when talking of the bblayers.conf file or the BBLAYERS variable. The same file also contains BBPATH = "${TOPDIR}" which is at least dynamically assigned and not that far away from the root yotco directory.

My best guess is that the bblayers.conf file is specific to the system it is being run on. That would make it unsuitable for sharing between developers via source control and the absolute paths would force people to edit the file whenever they received a copy. That did not seem like a very good reason though, hence the question.

Did you ever figure out how to do it? Sitting in the same boat right now :) — Mario Tacke, Jan 24 '17 at 00:19
@MarioTacke no I did not. I think the answer is that the `bblayers.conf` file is intended to be specific to a user on a machine and only temporary. Once you have sorted out what layers you want, you should make a recipe that replaces most of the `bblayers.conf` contents. You can use `${HOME}`, `${TOPDIR}` and a few other substitutions but not relative paths. — TafT, Jan 24 '17 at 09:15
No one is answering the question. I am not asking "How do I use relative paths" I am asking "Why is their use blocked". Looking back at this now I am guessing that the answer is for efficiency. For an absolute path you can navigate directly to the target file. For a relative path you have to work out a present working directory, work out what is N lots of `../`above it and move there, then move back down another directory path. That sounds like more work than go to /home/TafT/yocto/meta. — TafT, Apr 26 '17 at 07:34

score 14 · Answer 1 · edited May 23 '17 at 12:02

14

I found a way to use relative paths.

You can use inline python to traverse the file system. The following script uses the provided TOPDIR variable and then navigates to its parent via python's os.path api.

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

YOCTOROOT = "${@os.path.abspath(os.path.join("${TOPDIR}", os.pardir))}"

BBLAYERS ?= " \
  ${YOCTOROOT}/poky/meta \
  ${YOCTOROOT}/poky/meta-yocto \
  ${YOCTOROOT}/poky/meta-yocto-bsp \
"

BBLAYERS_NON_REMOVABLE ?= " \
  ${YOCTOROOT}/poky/meta \
  ${YOCTOROOT}/poky/meta-yocto \
"

References

OpenEmbedded's bitbake.conf
Advanced functionality with python
How do I get the parent directory in Python?

edited May 23 '17 at 12:02

Community

1
1

answered Jan 24 '17 at 18:00

Mario Tacke

5,378
3
30
49

How does YOCTOROOT differ from BBPATH or TOPDIR? I guess it gives you the effective ../ operation which would solve the issue and allow you to do `${YOCTOROOT}/my-layer/`. – TafT Jan 27 '17 at 09:53
mentioned solution is more flexible the one being initially proposed by poky/oe-init-build-env script looking for ##OEROOT## and replacing with the current path. because if the complete recipies tree will be moved another place, conf file will not be update with a new path automatically as here. – Oleg Kokorin Mar 14 '18 at 17:19
Is it possible to use the new path syntax (from Python 3.4) in inline Python? Is it then possible to enter multiple Python commands? https://stackoverflow.com/a/2860193/9704496 – Mario Palumbo Jun 20 '23 at 11:19

score 9 · Answer 2 · answered Apr 20 '17 at 00:03

9

I have managed to get "relative paths" in bblayers.conf files working by replacing

BBLAYERS ?= " \
  /home/username/poky/meta \
  ...

with

BBLAYERS ?= " \
  ${TOPDIR}/../meta \
  ...

I guess one caveat with this approach is that I'm relying on the meta-XXX layer directories always being in the parent folder of TOPDIR. This seems to be the case with the default way of using yocto, but might not be so for more customized build setups.

answered Apr 20 '17 at 00:03

gbmhunter

1,747
3
23
24

1

Sounds like you managed to get the thing I probably failed to do to work. Maybe the tools have updated to allow this behaviour now. Thank you for your answer, but it does not address the question of _why_ they were blocked. Although I suppose it demonstrates that now they may not be blocked any more so the answer could be that it was an mistaken without a reason. – TafT Apr 26 '17 at 07:40

score 3 · Answer 3 · answered Aug 11 '16 at 09:03

3

You can use relative paths in bblayers.conf.

There is probably this line in your bblayers.conf:

BBPATH = "${TOPDIR}"

When you want to find out this variable's content, you will probably find the top-level directory of your build directory:

bitbake -e | grep ^TOPDIR
# searches for bitbake variables

Inside this directory you could create a layer meta-test and add it in bblayers.conf with a relative path:

BBLAYERS ?= " \
  meta-test \
  [...]
  "

So the answer on your question why there are absolute paths in bblayers.conf is that you can place your build directory anywhere on the system and not dependant from Yocto.

Relative Paths to Layers must always be relative to the build directory.

answered Aug 11 '16 at 09:03

h0ch5tr4355

2,092
4
28
51

Your answer seems to state that paths relative to a build directory are possible. Nothing says that relative paths can actually be used. If the Yocto (or bitbake) system takes only an Absolute path for ${TOPDIR} and then uses string manipulation to append the BBLAYERS paths, that is still using only absolute paths. Are you saying that the ability to put a build directory anywhere is maintained by the requirement of an absolute path to it? Is there a source for these statements. – TafT Sep 23 '16 at 09:37
I don't understand this. A relative path always needs a reference point. This sticks in the term "relative". In this point it would be BBPATH. You can access any file on your filesystem via a relative path provided that the security settings fit. – h0ch5tr4355 Sep 23 '16 at 11:33
If I enter `./` or `../../` in the paths I use they fail. It is that sort of relative path that I am asking about. I can see that using the bitbake system it is possible to have some times in paths relative to other things within the bitbake ecosystem. I am asking why I cannot have a path defined as `../foo/bar.txt` inside the bblayers.conf file when it seems possible in most of the other files I try it in. – TafT Sep 23 '16 at 12:16

Étienne · Accepted Answer · 2019-08-29T10:23:55.847

All the existing answers are answering "how to use relative paths" but the question is "why are absolute paths used". As far as I know the "why" is simple, it is done that way so that the build directory can be moved anywhere on the filesystem. Think about it: you can source poky/oe-init-build-env from anywhere on the filesystem and the build directory will be created there, so relying on paths relative to the build directory is very fragile.

Edit:

maybe this is clearer, I think you are assuming that the file bblayers.conf is always in poky/build/conf/bblayers.conf and that you can therefore use a path like ../../meta-layer-foo to refer so some layer which would be in poky/meta-layer-foo, but the layer will not be found if I instantiate "build" in another path poky/foo/bar:

etienne@ubuntu:~/repos/poky-tx2$ mkdir -p foo/bar
etienne@ubuntu:~/repos/poky-tx2$ cd foo/bar/
etienne@ubuntu:~/repos/poky-tx2/foo/bar$ ls
etienne@ubuntu:~/repos/poky-tx2/foo/bar$ source ../../oe-init-build-env 
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.

You had no conf/bblayers.conf file. This configuration file has therefore been
created for you with some default values. To add additional metadata layers
into your configuration please add entries to conf/bblayers.conf.

The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
    http://yoctoproject.org/documentation

For more information about OpenEmbedded see their website:
    http://www.openembedded.org/


### Shell environment set up for builds. ###

You can now run 'bitbake <target>'

Common targets are:
    core-image-minimal
    core-image-sato
    meta-toolchain
    meta-ide-support

You can also run generated qemu images with a command like 'runqemu qemux86'
etienne@ubuntu:~/repos/poky-tx2/foo/bar/build$ ls
conf

Do you mean that for most development the directories referred to inside `./build/conf/bblayers.conf` are likely to be universal for all projects on a machine? So the paths refered to in bblayers.conf will be indepndant of the build directory as the build directroy will move the other things never will? — TafT, Aug 29 '19 at 09:58
What I mean is that when you create a project, you do not know where people will instantiate the build directory, so using relative paths cannot work unless you make an assumption about the location of "build", which will then break if people call "source oe-init-build-env" from some other path (the "build" directory gets created as a subdirectory of the current directory when you call this script, so this can be anywhere on your filesystem). — Étienne, Aug 29 '19 at 10:19
The need to source the contents of the directory from anywhere on the filesystem could be the reason. It was not that clear in your first response but it is clearer now. — TafT, Sep 02 '19 at 12:30

Kamal Pandey · Answer 5 · 2018-08-23T13:50:03.500

I am working with Rocko version and my bblayers.conf file also doesn't support relative paths I tried to change the bblayers.conf file by using TEMPLATECONF variable. TEMPLATECONF variable points to the directory containing bblayers.conf.sample, layer.conf, and local.conf.sample. I exported the TEMPLATECONF variable to get the desired bblayers.conf and local.conf files in the build directory but in my bblayers.conf.sample the BBLAYERS variable was set by a relative path as shown below:

BBLAYERS ?= " \
  ##OEROOT##/meta \
  ##OEROOT##/../meta-xilinx \
  ##OEROOT##/../meta-xilinx-tools \
  ##OEROOT##/../meta-openembedded/meta-oe \
  ##OEROOT##/../meta-openembedded/meta-perl \
  ##OEROOT##/../meta-openembedded/meta-python \
  ##OEROOT##/../meta-openembedded/meta-multimedia \
  ##OEROOT##/../meta-openembedded/meta-networking \
  ##OEROOT##/../meta-openembedded/meta-filesystems \
  ##OEROOT##/../meta-openembedded/meta-webserver"

but it doesn't seem to work.OEROOT variable was not able to set the correct paths. One reason could be that as oe-init-build-env script ends it unset the OEROOT variable. Although if you manually export OEROOT variable to your required value it may help. However, when I changed from OEROOT to TOPDIR variable it worked like a charm as shown below:

BBLAYERS ?= " \
  ${TOPDIR}/../meta \
  ${TOPDIR}/../meta-poky \
  ${TOPDIR}/../meta-skeleton \
  ${TOPDIR}/../meta-selftest \
  ${TOPDIR}/../meta-yocto-bsp \
  ${TOPDIR}/../../meta-xilinx/meta-xilinx-bsp \
  ${TOPDIR}/../../meta-xilinx/meta-xilinx-contrib \
  ${TOPDIR}/../../meta-xilinx-tools \
  ${TOPDIR}/../../meta-openembedded/meta-oe \
  ${TOPDIR}/../../meta-openembedded/meta-perl \
  ${TOPDIR}/../../meta-openembedded/meta-python \
  ${TOPDIR}/../../meta-openembedded/meta-multimedia \
  ${TOPDIR}/../../meta-openembedded/meta-networking \
  ${TOPDIR}/../../meta-openembedded/meta-filesystems \
  ${TOPDIR}/../../meta-openembedded/meta-webserver"

Which probably make me think that unsetting of OEROOT variable by the oe-root-init-env script caused the problem. Also if somebody finds a better solution please respond.

You can use `##OEROOT##` provided you use it in `bblayers.conf.sample` and you update your `templateconf.cfg` to point to your `conf` directory. I've added a separate answer to provide details about this strategy. — Yves Lhuillier, Nov 30 '21 at 11:27

score 0 · Answer 6 · answered Nov 30 '21 at 11:25

As you already mentioned in your self-comment, bblayers.conf is

intended to be specific to a user on a machine and only temporary.

IMHO, the idea is that bblayers.conf should never be distributed amongst developers. Nevertheless, if the absolute path of layers is something specific to each developers' system installation, it seems sound to keep the actual layer list as part of the project.

Inspired by Kamal Pandey's answer, I came up with a solution leveraging template file engine behind the oe-init-build-env script.

By changing the content of conf/templateconf.cfg to conf (instead of meta-poky/conf by default), you instruct oe-init-build-env to add any missing files in your configuration directory from its .sample counterpart (now located in your conf directory).

This way, renaming your bblayers.conf to bblayers.conf.sample lets you use the OEROOT variable (which is unfortunately no longer available after oe-init-build-env invocation).

# POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf                                                                                                                                      
# changes incompatibly                                                                                                                                                                                            
POKY_BBLAYERS_CONF_VERSION = "2"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \                                                                                                                                                                                                   
  ##OEROOT##/meta \                                                                                                                                                                                               
  ##OEROOT##/meta-poky \                                                                                                                                                                                          
  ##OEROOT##/meta-yocto-bsp \                                                                                                                                                                                     
  ${TOPDIR}/meta-tensorflow-lite \                                                                                                                                                                                
  "

With this bblayers.conf.sample file, you can now use the ##OEROOT## substitution variable to refer to file relative to your poky installation directory. By removing bblayers.conf (or not versioning/ditributing it), you ensure that sourcing oe-init-build-env will regenerate bblayers.conf with correct absolute values.

Why does the yocto bblayers.conf file use absolute paths?

6 Answers6

References