Jenkins Job-Builder: How to correctly include job-templates from external file?

Question

I am investigating using Jenkins Job-Builder (from OpenStack) as our means of managing jenkins job configurations. In doing so I am trying to figure out the right (best?) way to include a job-template from an external file using the !include custom tag.

In the current use case we will basically have one template that is going to be used by a LOT of job. Each job is going to need to exist in its own file for reason that are out of scope here.

So far I have gotten this to work:

job-template.yml

name: 'pre-build-{proj}-{repo}'
project-type: freestyle
... etc ...

job-1.yml

- job-template:
    !include job-template.yml

- project: 
    name: job-1
    proj: my-proj
    repo: my-repo
    jobs:
      - 'build-config-{proj}-{repo}'

---

This seem wrong because the template definition gets split across both files and require needless duplication of the -job-template: line in every job file. I would like to get the following to work instead:

job-template.yml

- job-template:
    name: 'pre-build-{proj}-{repo}'
    project-type: freestyle
    ... etc ...

job-1.yml

!include job-template.yml

- project: 
    name: job-1
    proj: my-proj
    repo: my-repo
    jobs:
      - 'build-config-{proj}-{repo}'

The latter unfortunately results in a yaml parse error on the - project: line:

yaml.scanner.ScannerError: mapping values are not allowed here in "job-1.yml", line 3, column 10

Is there way to get the entire template definition into the template file? This will become particularly annoying if ever we need to pull in multiple templates from multiple files.

Answer 1

Jenkins-jobs takes a path argument which can be a directory holding your files (job-template.yaml, job-1.yaml and job-2.yaml. It will assemble them as a single YAML document, so you do not need to use !include. So you can write:

job-template.yaml

- job-template:
    name: 'pre-build-{proj}-{repo}'
    builders:
        - shell: 'echo building {proj} for {repo}'

job1.yaml

- project:
    name: job-1
    proj: my-proj
    repo: my-repo
    jobs:
        - 'pre-build-{proj}-{repo}'

job2.yaml

- project:
    name: job-2
    proj: my-other-proj
    repo: my-other-repo
    jobs:
        - 'pre-build-{proj}-{repo}'

That will generates two jobs with the following shell commands:

pre-build-my-other-proj-my-other-repo:

<command>echo building my-other-proj for my-other-repo</command>

pre-build-my-proj-my-repo:

<command>echo building my-proj for my-repo</command>

Assuming the files are in a directory config/ you can generate them all with:

jenkins-jobs test config/ -o /tmp/myjobs

Or use the name argument to filter the jobs that will be realized:

jenkins-jobs test config/ -o /tmp/myjobs '*my-proj*'
# Creates pre-build-my-proj-my-repo
# Skips pre-build-my-other-proj-my-other-repo