27

We use Jenkins for doing incremental builds of our project on each commit to the SCM. We would like to get separate builds for every single commit. However, the naive approach (setup SCM and use post-commit hooks to trigger a build) exhibits problem in the following scenario:

  • Build is triggered.
  • While build takes place (it can take up to several minutes) two separate commits to the SCM are made by two developers.
  • One new build is triggered. It receives changes from both of the commits, made during previous build.

This "race condition" complicates finding which one of the commits has broken the build/introduced warnings.

The currently employed solution is checking for changes in one job ("scheduler job") and triggering another job to do the actual checkout and build.

Are there any proper solutions to this problem?

Oleg
  • 271
  • 3
  • 3

6 Answers6

7

Not yet, there's a Feature Request covering this kind of build, but it's still open: Issue 673

Community
  • 1
  • 1
Daniel
  • 3,019
  • 1
  • 20
  • 21
2

Maybe it misses the point, but we have a pretty nice build process running here.

  • We use git as our source control system
  • We use gerrit as our review tool
  • We use the gerrit trigger to run builds on our jenkins server
  • We check for changes on the develop branch to run jenkins when a changeset is merged

In short the ideal developer day is like this

  1. developer 1 stars a new branch to do his changes, based on our main develop branch
  2. The developer 1 commits as often as he likes
  3. developer 1 thinks he finished his job, he combines his changes into one change and pushes it to gerrit
  4. A new gerrit change is created and jenkins tries to build exactly this change
  5. When there are no errors during the build, a review is made on this change
  6. When the review is submited, the changeset is merged into the develop branch of the main repository (No change is merged into the develop branch, without review)
  7. Jenkins builds the merged version to be sure, that there are no merge errors

no developer 2 joins the party and tries to do some work

the process is exactly the same both start working, in there branches. Developer 1 is faster and his changes are merged into the develop branch. Now, before developer 2 can publish his changes he has to rebase his changes on top of the changes made by developer 1.

So we are sure, that the build process is triggered for every change made to our codebase.

We use this for our C# development - on windows not on linux

MoJo2600
  • 409
  • 3
  • 10
1

As pointed out by someone in Issue 673 you could try starting a parametrized build with the parameter being the actual git commit you want to build. This in combination with a VCS commit hook.

cburgmer
  • 2,150
  • 1
  • 24
  • 18
1

I think what might help, is to set the Quiet Period (Jenkins > Manage Jenkins > Configure System) to 0 and the SCM Polling to a very short time. But even during that short interval there could be two commits. As of now Jenkins does not have the feature to split build into single builds on multiple SVN commit.

Here is a tutorial about that topic: Quiet Period Feature.

Community
  • 1
  • 1
Daniel Kutik
  • 6,997
  • 2
  • 27
  • 34
1

I don't believe what you'd like to do is possible. The "quiet period" mentioned by Daniel Kutik is actually used to tell Hudson/Jenkins how much time to wait, in order to allow other commits to the same project to be picked up. Meaning -- if you set this value to 60 seconds and you've made a commit, it will wait for a minute before starting a new build, allowing time for other commits to be picked up as well (during that one minute).

carlspring
  • 31,231
  • 29
  • 115
  • 197
  • I'm assuming that he has set a quiet 'high' quiet period. If he decreases the quiet period time the likelihood that two builds get 'merged' is also decreased. – Daniel Kutik Jul 12 '11 at 07:07
  • @Daniel Kutik: Yes, indeed. However, this does not really help in this case. – carlspring Jul 12 '11 at 07:15
  • @Daniel Kutik: No, the quiet period is set to zero and we tried both setting once-per-minute SCM polling and triggering the builds via URL. In both cases, several triggers are merged into one. – Oleg Jul 12 '11 at 09:22
  • Like I said, this is not supported. – carlspring Jul 12 '11 at 09:39
1

If you use the rule "NO COMMIT on a broken build‏" and take it to it's logical conclusion, you actually end up with "No commit on a broken build or a build in progress", in which case the problem you describe goes away.

Let me explain. If you have two developers working on the same project and both of them try to commit (or push if you're using DVCS). One of them is going to succeed and and they other will fail and need to update before the commit.

The developer who had to do the update knows from the commit history, that the other commit was recent and thus a build in progress (even if it hasn't checked out yet). They don't know if that build is broken yet of not, so the only safe option is to wait and see.

The only thing that would stop you from using the above approach is if the build takes so long, in which case you might find that your developers never get a chance to commit (it's always building). This is then a driver to split up your build into a pipeline of multiple steps, so that the Post Commit job takes no more than 5 minutes, but is ideally 1 minute.

Tom Howard
  • 6,516
  • 35
  • 58
  • With a DVCS you will still easily get this issue. Assume you are working on a local feature branch and then merge into master and push. There could be any number of commits that broke the build, admittedly you would know who broke it but it wouldn't help tracking down which commit did. – Luca Spiller Aug 30 '12 at 16:50
  • 1
    What you say is not necessarily true. Sometimes when the build is green but another build is in progress, multiple developers commit their changes to the repo. Next time Jenkins runs a job, it pulls the latest version of the repo and any of these changes may affect the outcome of the build. In big teams this can be a problem when the person that has broken the build should be responsible for fixing it. – Behrang Mar 14 '13 at 04:29