5

We are planning to setup Highly Available Jenkins setup in container platform using kubernetes. We are looking at setting up one Active master and another master in standby mode. Jenkins data volume is going to be stored in a global storage that is shared between the two master containers.

In case the active master is not available then requests should fail over to other master. And the slaves should be communicating only with active master.

How do we accomplish Jenkins HA setup in active/passive mode in kubernetes. please provide your suggestions.

We would like to achieve as shown in the diagram from below link

https://endocode.com/img/blog/jenkins-ha-setup_concept.png

P Ekambaram
  • 15,499
  • 7
  • 34
  • 59

3 Answers3

2

This contradicts with how one should IMHO run applications in Kubernetes. Active/passive is a concept for the past century.

Instead, configure a health check for the Jenkins Deployment. If that fails, Kubernetes will automatically kill the task and start a replacement (which will be available only a few seconds after detecting the active one being unhealthy).

StephenKing
  • 36,187
  • 11
  • 83
  • 112
  • the requirement is to run one jenkins master in active mode and other one in standby mode. – P Ekambaram Mar 13 '19 at 10:28
  • Badly formulated requirements.. maybe they should be better require that a replacement becomes active within X seconds instead of describing technical detail like active/passive implementation. – StephenKing Mar 13 '19 at 10:52
  • refer the design that we want to achieve. see the link in the question – P Ekambaram Mar 13 '19 at 10:59
  • But this is not made with K8s in mind, I guess. This solves it at the lower infrastructure layer. – StephenKing Mar 13 '19 at 11:06
  • @StephenKing to rely on K8S failover we should have persistent volume, so if jenkins instance get schedule on other node it should have data available, how can we sync data between nodes? external storage option is not available and only have three nodes and each node has spare disk. – ImranRazaKhan Dec 06 '21 at 03:58
  • you have K8s and don't have any PV abilities? – StephenKing Dec 06 '21 at 15:28
2

There have been active considerations to emulate active/passive setup for containers, but note that as a product feature this really is not a must have and hence not built in. This very well may be implemented as an OOB feature integration wherein you have to craft your applications to at least do the following:

  1. General leader election (for master selection and traffic routing, maybe a sidecar container to do elections and message routing)
  2. Make the liveness/readiness probe detection routines (and the failover logic) to patch all pods under failed paradigm to no longer be selected via any pod selector equation
  3. In the event of another failover, you will still have to ensure another patch of labels (and this time across the old and new pods) to update pods metadata aka labels

If you are looking for something bare minimal than, configuring liveness/readiness probes may just do the trick for you. As always, you should avoid getting into a practice of mass mutating pod labels with ad-hoc patches for role selection

https://github.com/kubernetes/kubernetes/issues/45300

Raunak Jhawar
  • 1,541
  • 1
  • 12
  • 21
0

If you are OK with an opinionated framework , then JenkinsX may help you. It comes by default with the features as required by you.

arun
  • 1