51

Q1: Whats the difference between

concurrent = 3

[[runners]]
  ..
  executor = "shell"

and

concurrent = 3

[[runners]]
  ...
  executor = "shell"

[[runners]]
  ...
  executor = "shell"

[[runners]]
  ...
  executor = "shell"

Q2: Does it makes sense, to...

have 3 executors (workers) of same type on a single runner with global concurrent = 3? Or can single executor with global concurrent = 3 do multiple jobs in parallel safely?

Q3: How they're related...

runners.limit with runners.request_concurrency and concurrent

Thanks

Community
  • 1
  • 1
Slimer
  • 1,123
  • 2
  • 10
  • 22

1 Answers1

77

Gitlab's documentation on runners describes them as:

(...) isolated (virtual) machines that pick up jobs through the coordinator API of GitLab CI

Therefore, each runner is an isolated process responsible for picking up requests for job executions and for dealing with them according to pre-defined configurations. As an isolated process, each runner have the capability of creating 'sub-processes' (also called machines) in order to run jobs.

When you define in your config.toml a [[runner]] section, you're configuring a runner and setting how it should deal with job execution requests. In your questions, you mentioned two of those "how to deal with job execution request"' settings:

  1. limit: "Limit how many jobs can be handled concurrently". In other words, how many 'sub-processes' can be created by a runner in order to execute jobs simultaneously;
  2. request_concurrency: "Limit number of concurrent requests for new jobs from GitLab". In other words, how many job execution requests can a runner take from GitLab CI job queue simultaneously.

Also, there are some settings that apply to a machine globally. In your question you mentioned one of them:

  1. concurrent: "Limit how many jobs globally can be run concurrently. This is the most upper limit of number of jobs using all defined runners". In other words, it limits the maximum amount of 'sub-processes' that can run jobs simultaneously.

Thus, keeping in mind the difference between a runner its sub-processes and also the difference between specific runner settings and global machine settings:

Q1:

The difference is that in your 1st example you have one runner and in your 2nd example you have three runners. It's worth mentioning that in both examples your machine would only allow running 3 jobs simultaneously.

Q2:

Not only a single runner can run multiple jobs concurrently safely but also is possible to control how many jobs you want it to handle (using the aforementioned limit setting).

Also, there is no problem to have similar runners running in the same machine. How you're going to define your runner's configurations is up to you and your infrastructure capabilities.

Also, please notice that an executor only defines how to run your job. It isn't the only thing that defines a runner and it isn't a synonymous for "worker". The ones working are your runners and their sub-processes.

Q3:

To summarize: You can define one or many workers at the same machine. Each one is an isolated process. A runner's limit is how many sub-processes of a runner process can be created to run jobs concurrently. A runner's request_concurrency is how many requests can a runner handle from the Gitlab CI job queue. Finally, setting a value to concurrent will limit how many jobs can be executed at your machine at the same time in the one or more runners running in the machine.

References

For better understanding, I really recommend you read about Autoscaling algorithm and parameters.

Finally, I think you might find this question on how to run runners in parallel on the same server useful.

Aurora Wang
  • 1,822
  • 14
  • 22
  • 8
    Thanks for your answer. To verify if i understand it correctly: if I have concurrent=1 and 3 workers, then only 1 of them can take the job, while other 2 will be idle. on the other hand, when i have concurrent = 3, and 3 workers each with limit = 1, then single machine can run 3 jobs in parallel, but each worker can take only 1 job. Correct? – Slimer Mar 15 '19 at 14:54
  • 1
    how can be concurrent to be set programmaticaly? – Ivan Bryzzhin Jul 06 '21 at 12:21
  • 7
    The difference between `limit` and `request_concurrency` is still unclear for me. – Dzenly Aug 10 '21 at 08:58
  • 2
    If only one runner defined on this machine, and `concurrent=3, limit=3`, what's the difference for `request_concurrency=1 or 3`? – heLomaN Sep 16 '21 at 07:56
  • The effective limit on the number of concurrent jobs executed by a single runner, not accounting for the global `concurrent` limit, appears to be `limit == 0 ? request_concurrency : min(limit, request_concurrency)`. Why in the world this configuration is so confusing and split into two different settings when one would obviously suffice, I cannot say. (default value for `limit` is `0` and `request_concurrency` is `1` so just ignore `limit` and set `request_concurrency` ) – Paul Wheeler Aug 30 '22 at 19:34
  • @Dzenly I think this should be added to the answer. I don't know if there is any report, but at least on gitlab-runner `14.3.2` `limit` is used to work around a bug: if you set `request_concurrency = 1`, it actually will be ignored, I presume in preference of the global `concurrent` setting. In my tests the runner with `request_concurrency = 1` will happily take 2 jobs simultaneously *(which is usually not something you'd want, there's usually a reason for limiting the concurrency, such as access to a shared resource)*. To make it actually work you have to additionally set `limit = 1`. – Hi-Angel Sep 01 '22 at 22:11