13

I'm trying to get the following workflow running in Jenkins, in parallel. So for example, both A and B running at the same time. As soon as A finishes run A2 and A3 at the same time, etc...

It seems like a pretty common use case but most plugins I tried do no support more than 1 level deep, which is the case with the A branch below. Join plug-in doesn't seem helpful here.

I read about using the Promotion plugin but I'm a little mystified on what to fingerprint/artifacts to archive to make this work.

Any clue on how to make this simple build pipeline work?

flow

Community
  • 1
  • 1
vdsf
  • 1,608
  • 2
  • 18
  • 22

6 Answers6

7

As jgritty pointed out you could use the Build flow plugin. In order to get the type of parallel execution you want you could run something equivalent to the following build flow script:

buildTrigger = build("Trigger")

parallel(
    {
        buildA = build("A")
        buildA1 = build("A1")
        parallel(
            {
                buildA2 = build("A2")
            },
            {
                buildA3 = build("A3")
            },
        )
    },
    {
        buildB = build("B")
        buildB1 = build("B1")
    },
)

buildResults = build("GatherResult")

In this script the first parallel block takes care of the A and B branches. Once in a branch each build is sequential until you add more parallel blocks.

Community
  • 1
  • 1
Petrik
  • 1,961
  • 13
  • 23
  • 1
    I see warning -> Deprecated: Users should migrate to https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Plugin. not sure how to really do this – Vikas Thange May 07 '21 at 14:59
7

There is a solution which works great: the Multi-Job plugin. It does exactly what you want. With The MultiJob Plugin you can split your job into phases which run serially. Within each phase, the jobs run in parallel.

For simplicity I'm going to assume that A and B are "compilation jobs" you wish to run in parallel. Further assume there may be an A-Test (and its children) and B-Test which are also separate jobs.

You create the multijob as follows: New Item -> Multijob Project

In the project you create two phases (Add build step "MultiJob Phase"). The first will be COMPILE and you would add Phase Job's "A" and "B".

You can change the options for the COMPILE phase jobs so that if either fails, the whole phase aborts (the default), or allow the job to continue.

MultiJob Setup

Next you add another MultiJob Phase Build step for TEST and add "A-Test" and "B-Test" to that. Remember that A, B, A-Test, and B-Test are all separate jobs that can also be run individually if desired.

That's it. When the job runs it will contain links to the child jobs so you can see what happened with the sub-jobs.

Steven the Easily Amused
  • 730
  • 9
  • 23
1

I believe this may be the plugin you need.

jgritty
  • 11,660
  • 3
  • 38
  • 60
  • The build flow plugin seems to support only 1 level deep as well. Looking at the examples, if you have a parallel block X, followed by another block Y, Y will only start once all jobs in X have finished. So it's similar to using a 1-level deep join in the Join plugin, or the Multijob plugin. – vdsf Nov 13 '12 at 18:36
1

I had a similar problem, and the Promoted Builds Plugin was a better solution for me than the Build Flow Plugin that others have advised.

A build promotion job can be added to an existing job by ticking the check-box "Promote builds when...". You can configure the promotion to occur when named downstream projects are complete by checking "When the following downstream projects build successfully" under Criteria (assuming that all downstream projects are related by fingerprints).

At the promotion step, you can trigger emails of summary results or trigger a further downstream job.

In this case, you might add the promotion job to the Trigger projects, and depend on projects A2 A3 and B1 completing successfully.

laffoyb
  • 1,540
  • 3
  • 22
  • 35
0

Use Build Other projects in Post-Build Section. Say start A2,A3 on completion of A1. Increase number of executors in Manage Hudson --> Configure system to appropriate number.

Vishal Pahuja
  • 61
  • 3
0

We have a set up similar to what 'Steven the Easily Amused' suggested. Except I set up the "A" and "B" paths as their own Multi-Job stack and another Multi-Job set up as the "Trigger". Where both the "A" and "B" stack run in parallel. This way they don't directly have a tight dependency on each other. You can also aggregate your results depending on if one/both stacks fail. You can also set up the "Trigger" to collect parameters and pass then to both stacks.

RobertoS
  • 61
  • 4