16

In most Jenkins' examples the SCM poll value is designated */15 * * * *, ie. poll SCM every 15 minutes. That's fine when you have hundreds of jobs, but not if you have thousands of jobs as it results in thousands of request to the SCM (Subversion in this case) server at 15, 30, 45, and 0 minutes past the hour.

Is there some way to randomize the polling value in Jenkins so as to avoid the above scenario?

On the Jenkins master configuration screen there's a value Max # of concurrent polling. Should this be set (and to what value) to avoid the above scenario?

Yuri
  • 4,254
  • 1
  • 29
  • 46
Michael Neale
  • 19,248
  • 19
  • 77
  • 109
  • Late addition related to the issue, not the question: Jenkins allows you to limit the number of workers polling the SCM. This will resolve your issue without the need for randomization. See http://jenkins-ci.361315.n4.nabble.com/of-Executors-affect-polling-too-td3622606.html – Omri Spector Jun 12 '13 at 13:30

4 Answers4

29

Use H instead of * and Jenkins will randomly distribute the polling. Note that at present a good syntax has not been found for a frequency differing from once per hour/day/etc, so

H * * * *

will poll once per hour at a a pre-determined random minute.

H H * * *

will poll once per day at a pre-determined random hour and minute

H H H * *

will poll once per week

0 H * * *

will poll on the hour but once per day at a pre-determined random hour.

Keep in mind that you are allowed multiple cron lines and any will match, so until a good syntax for sub-hour frequency has been settled on, you can get closer (on average) with something like

H * * * *
H * * * *
H * * * *
H * * * *
H * * * *
H * * * *

will give you on average 15 minutes between polling (yes there will be 6 pollings per hour, but that is to give a good chance that at least one polling will fall in each quarter hour)

If you have good suggestions for a syntax distributing within the hour please respond to this thread:

https://groups.google.com/forum/?fromgroups#!jenkinsci-users/VghEjfygWuw/PuIG1o7u1GQJ%5B1-25%5D

Update (April 2013)

Jenkins 1.510 and newer includes a new syntax to allow specifying distributions within the hour

Jesse Glick
  • 24,539
  • 10
  • 90
  • 112
Stephen Connolly
  • 13,872
  • 6
  • 41
  • 63
  • +1 for the `1.510` update. Too bad the latest LTS is `1.509.1`. Almost enough of a reason to upgrade to a non-LTS release. – Snekse May 09 '13 at 14:57
2

If you set max# of concurrent polling to something between 1 and 3 then the polling requests will simply queue up and be processed serially (or at most 3 in parallel).

Given that all that is involved in the polling for SVN is effectively

svn info branch-url

They should all complete quickly, but at least you will have limited the parallel requests.

Michael Neale
  • 19,248
  • 19
  • 77
  • 109
1

@Stephen Connolly's solution is likely a good solution for this.

If you really have thousands of jobs to worry about, consider setting the timing based on some part of the job name (Assuming your job names are reasonably distributed).

For example, if your job starts with 'B', set the timings to */2. If it starts with 'C', set it to */3 etc. ('A' would need something other than 1 though).

It's not a terrific solution, but if you're really looking at managing that many jobs it might be a work-around until you can find a better solution.

Peter Bernier
  • 8,038
  • 6
  • 38
  • 53
0

H H H * *

I think above will poll once per month not per week

abhinav
  • 411
  • 6
  • 7