4

I am currently working on the following scenario

I am trying to setup a container in OpenShift that runs a Jenkins that is itsself able to run docker to make use of declarative pipelines where the build is running in it's own docker container. This basically makes it necessary to install and run docker inside this container.

I have been working on it on quite some time now. Checked dozens of posts and threads online but I have not been able to accomplish it. Basically I got so far

  • I can install docker in my container (from the baseimage openshift/jenkins-2-centos7:latest)
  • I can't get docker to run as this makes use of systemctl which

Now I read that systemctl is not working inside docker containers or at least highly unrecommended as it interferes with the PID 1 in the system. Without

systemctl start docker

that will leave me with docker beeing unable to connect with the daemon (as expected) and the error message

Can't connect to docker daemon. Is 'docker -d' running on this host?

So I tried to set up the daemon myself using

the follwoing in my Dockerfile

RUN usermod -aG docker $(whoami)
RUN dockerd -H unix:///var/run/docker.sock

which will also not work telling me that cgroups cannot be mounted. After some more research I found that this could be handled with the cgroupfs-mount script from

https://github.com/tianon/cgroupfs-mount/tree/master

But also here I got no luck leaving me with the following error

Error starting daemon: Error initializing network controller: error obtaining controller instance: failed to create NAT chain DOCKER: iptables failed: iptables -t nat -N DOCKER: iptables v1.4.21: can't initialize iptables table `nat': Permission denied (you must be root) Perhaps iptables or your kernel needs to be upgraded.

Now after hours I am out of ideas. Does anyone have an idea how to make docker work inside of OpenShift? Would be really greatful

grdryn
  • 1,972
  • 4
  • 19
  • 28
relief.melone
  • 3,042
  • 1
  • 28
  • 57

3 Answers3

2

You have this article by @jpetazzo, from Docker team, about Docker In Docker (DinD):

article:

The primary purpose of Docker-in-Docker was to help with the development of Docker itself. Many people use it to run CI (e.g. with Jenkins), which seems fine at first, but they run into many “interesting” problems that can be avoided by bind-mounting the Docker socket into your Jenkins container instead.

DinD Repo:

This work is now obsolete, thanks to the combined efforts of some amazing people like @jfrazelle and @tianon, who also are black belts in the art of putting IKEA furniture together.

If you want to run Docker-in-Docker today, all you need to do is:

docker run --privileged -d docker:dind

So here is an article using another approach to build docker containers with Jenkins inside a docker container:

docker run -p 8080:8080 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  --name jenkins \
  jenkins/jenkins:lts

So you may want to adapt one of this solutions to your OpenShift scenario. I hope it solves your issue.

Exadra37
  • 11,244
  • 3
  • 43
  • 57
  • thanks for the input. ill work through it. also hope this clarifies things. pretty steep learning curve with openshift and at the same time beeing at most an intermediate with linux in general – relief.melone Jan 31 '19 at 18:40
  • well it definately sounds like a great approach to bind-mount the docker.sock. I'll test that one out. I hope it won't die on OpenShift-restrictions – relief.melone Jan 31 '19 at 18:48
  • let me know how it went ;) – Exadra37 Jan 31 '19 at 19:18
  • not that good so far. my problem is, that i cannot run docker -v in OpenShift. Found another interesting thing here https://lists.openshift.redhat.com/openshift-archives/users/2016-March/msg00216.html but it's pretty old and currently my mount for docker.sock is not working. No error, but the file just won't get mounted. I'll keep trying and update when something comes of it – relief.melone Feb 01 '19 at 08:10
  • I am not experienced with OpenShift... Try `docker --mount=...` instead, if it works I will update my answer with it, read [here](https://docs.docker.com/storage/bind-mounts/) about it. – Exadra37 Feb 01 '19 at 08:40
  • The problem is OpenShift runs docker. I don't have access to the arguments and addional to that it seams that I cannot set my container to run in privileged mode which prevents me from mounting docker.sock. I'll keep on trying to figure it out and come back once i have ;) – relief.melone Feb 01 '19 at 10:04
2

I am trying to setup a container in OpenShift that runs a Jenkins that is itsself able to run docker to make use of declarative pipelines where the build is running in it's own docker container. This basically makes it necessary to install and run docker inside this container.

I don't think your conclusion here is the only possibility, and what I'll describe below is an easier approach to get what (I think) you want! :) If there are any other use cases that you have than these 3 I'll describe, let me know and I'll try to update to cover them:

  • Pipelines running in their own containers
  • Running additional containers from Pipelines
  • Building container images from Pipelines

Pipelines running in their own containers

For this case, there's the excellent Kubernetes plugin.

With this plugin, you add a Kubernetes/OpenShift cloud to the Jenkins global config. This can either be the one in which Jenkins is running (if you use the Jenkins image provided by OpenShift, this gets added by default at least), or an external cluster.

Inside that configuration, you can define PodTemplates (again, there are a couple of examples provided in the Jenkins image provided by OpenShift), or you can specify that in your pipeline directly also I think. When your pipeline requests a node/agent with a label that matches one of these (and there are no long-running agents that match), then a pod will be created from that template, and your pipeline execution will happen inside a container in that. Once it's no longer needed, it will be deprovisioned again.

Here are the pipeline steps exposed by this plugin: https://jenkins.io/doc/pipeline/steps/kubernetes/

Running additional containers from Pipelines

As part of your pipeline, you may want to run some tests, and those may expect to be able to interact with e.g. a database. You can create resources for that in your OpenShift project (e.g. a Deployment & expose it with a Service), and tear them down after. The openshift-client plugin is very useful here and has docs on how to interact with OpenShift.

Building container images from Pipelines

If your goal is to build container images from pipelines, remember that OpenShift also exposes this capability (depending on the security configuration) through Builds. Just like in the previous section, you can use the openshift-client plugin to create and trigger builds.

For more information on the Jenkins image that's maintained by OpenShift (and generally how to do useful things in Jenkins on OpenShift), there's this dedicated page in the OpenShift docs.

grdryn
  • 1,972
  • 4
  • 19
  • 28
  • Thanks. I will tage a look at the kubernetes plugin. Also sounds promising. The 3rd option is actually the way i do it atm. My problem here is that once i start the containers via pipeline to run e.g. integration tests I don't get the results presented back if it was successfull etc. – relief.melone Feb 01 '19 at 18:30
0

You'll need a privileged pod running jenkins wich mounts the openshift node docker socket. This is a bad idea as jenkins'll launch container outside kubernetes semantics and control.

Why do not use s2i service shipped with openshift ?

Regards.

mdaguete
  • 387
  • 2
  • 4
  • priviledged containers are unfortunately no option for me. the reason it want to use jenkins and docker on that pod as well is because i want to automate the deployment of jenkins as well as being able to run isolated build tasks on that jenkins in their specific environment – relief.melone Jan 31 '19 at 18:39