1

On my OS X host, I'm using Docker CE (18.06.1-ce-mac73 (26764)) with Kubernetes enabled and using Kubernetes orchestration. From this host, I can run a stack deploy to deploy a container to Kubernetes using this simple docker-compose file (kube-compose.yml):

version: '3.3'
services:
  web:
    image: dockerdemos/lab-web
    volumes:
      - "./web/static:/static"
    ports:
      - "9999:80"

and this command-line run from the directory containing the compose file:

docker stack deploy --compose-file ./kube-compose.yml simple_test

However, when I attempt to run the same command from my Jenkins container, Jenkins returns:

this node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again

I do not want the docker client in the Jenkins container to be initialized for a swarm since I'm not using Docker swarm on the host.

The Jenkins container is defined in a docker-compose to include a volume mount to the docker host socket endpoint:

version: '3.3'
services:
  jenkins:
    # contains embedded docker client & blueocean plugin
    image: jenkinsci/blueocean:latest
    user: root
    ports:
      - "8080:8080"
      - "50000:50000"
    volumes:
      - ./jenkins_home:/var/jenkins_home
      # run Docker from the host system when the container calls it.
      - /var/run/docker.sock:/var/run/docker.sock
      # root of simple project
      - .:/home/project
    container_name: jenkins

I have also followed this guide to proxy requests to the docker host with socat: https://github.com/docker/for-mac/issues/770 and here: Docker-compose: deploying service in multiple hosts.

Finally, I'm using the following Jenkins definition (Jenkinsfile) to call stack to deploy on my host. Jenkins has the Jenkins docker plug-in installed:

node {
    checkout scm

    stage ('Deploy To Kube') {
        docker.withServer('tcp://docker.for.mac.localhost:1234') {
            sh 'docker stack deploy app --compose-file /home/project/kube-compose.yml'
        }
    }      
}

I've also tried changing the withServer signature to:

docker.withServer('unix:///var/run/docker.sock')

and I get the same error response. I am, however, able to telnet to the docker host from the Jenkins container so I know it's reachable. Also, as I mentioned earlier, I know the message is saying to run swarm init, but I am not deploying to swarm.

I checked the version of the docker client in the Jenkins container and it is the same version (Linux variant, however) as I'm using on my host:

Docker version 18.06.1-ce, build d72f525745

Here's the code I've described: https://github.com/ewilansky/localstackdeploy.git

Please let me know if it's possible to do what I'm hoping to do from the Jenkins container. The purpose for all of this is to provide a simple, portable demonstration of a pipeline and deploying to Kubernetes is the last step. I understand that this is not the approach that would be taken anywhere outside of a local development environment.

ewilan
  • 638
  • 6
  • 16

1 Answers1

1

Here is an approach that's working well for me until the Jenkins Docker plug-in or the Kubernetes Docker Stack Deploy command can support the remote deployment scenario I described.

I'm now using the Kubernetes client kubectl from the Jenkins container. To minimize the size increase of the Jenkins container, I added just the Kubernetes client to the jenkinsci/blueocean image that was built on Alpine Linux. This DockerFile shows the addition:

FROM jenkinsci/blueocean
USER root
RUN curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.12.0/bin/linux/amd64/kubectl
RUN chmod +x ./kubectl
RUN mv ./kubectl /usr/local/bin/kubectl
RUN mkdir /root/.kube
COPY kube-config /root/.kube/config

I took this approach, which added ~100 mb to the image size rather than getting the Alpine Linux Kubernetes package, which almost doubled the size of the image in my testing. Granted, the Kubernetes package has all Kubernetes components, but all I needed was the Kubernetes client. This is similar to the requirement that the docker client be resident to the Jenkins container in order to run Docker commands on the host.

Notice in the DockerFile that there is reference to the Kuberenetes config file:

kube-config /root/.kube/config

I started with the Kubernetes configuration file on my host machine (the computer running Docker for Mac). I believe that if you enable Kubernetes in Docker for Mac, the Kubernetes client configuration will be present at ~/.kube/config. If not, install the Kubernetes client tools separately. In the Kubernetes configuration file that you will copy over to the Jenkins container via DockerFile, just change the server value so that the Jenkins container is pointing at the Docker for Mac host:

    server: https://docker.for.mac.localhost:6443

If you're using a Windows machine, I think you can use docker.for.win.localhost. There's a discussion about this here: https://github.com/docker/for-mac/issues/2705 and other approaches described here: https://github.com/docker/for-linux/issues/264.

After recomposing the Jenkins container, I was then able to use kubectl to create a deployment and service for my app that's now running in the Kubernetes Docker for Mac host. In my case, here are the two commands I added to my Jenkins file:

 stage ('Deploy To Kube') {
    sh 'kubectl create -f /kube/deploy/app_set/sb-demo-deployment.yaml'
}
stage('Configure Kube Load Balancer') {
    sh 'kubectl create -f /kube/deploy/app_set/sb-demo-service.yaml'
}

There are loads of options for Kubernetes container deployments. In my case, I simply needed to deploy my web app (with replicas) behind a load balancer. All of that is defined in the two yaml files called by kubectl. This is a bit more involved than docker stack deploy, but achieves the same end result.

ewilan
  • 638
  • 6
  • 16