Allow docker app to access kubectl pointed cluster

Question

My local system's kubectl is currently pointing to my digitalocean kubernetes cluster. I wrote an app in Rust that can list the pods on my cluster and their CPU usage, which is working fine on local system.

Now, I dockerized this application, so now it is running as a container.

How can I configure my docker-compose in such a way, so that it can access local system's kubectl ?

I tried this on the basis of suggested solutions:

version: "3.3"

services:
  custom-tool:
    container_name: custom-tool
    image: myimage:v16
    restart: always
    command: ./main-thread
    environment:
      - KUBECONFIG=/root/config
    volumes:
      - /home/keval/.kube/config:/root/config

But, no luck yet !

@GerardGarcia, the app uses kubernetes Rust SDK, and the SDK functions use current context pointed by kubectl (if none specified explicitly). — Keval Bhogayata, Jun 01 '22 at 12:12
And how is your kubectl configured? How does it know where is your Kubernetes cluster and how to authenticate? — Gerard Garcia, Jun 01 '22 at 12:54

Gerard Garcia · Answer 1 · 2022-06-02T12:06:01.767

2

It seems like you can create your own configuration programmatically, have it read from ~/.kube/config or from env variables: https://docs.rs/kube/0.73.0/kube/struct.Config.html

The easiest option you have is to have your local .kube/config available inside your container by using a bind mount (most likely at /root/.kube/config).

It will look like this:

version: "3.3"

services:
  custom-tool:
    container_name: custom-tool
    image: myimage:v16
    restart: always
    command: ./main-thread
    volumes:
      - type: bind
        source: /home/keval/.kube/config
        target: /root/.kube/config

edited Jun 02 '22 at 12:06

answered Jun 01 '22 at 12:29

Gerard Garcia

1,554
4
7

I doubt this will work, because docker environment does not have a kubernetes installed. and therefore, binding a config file will not help ! – Keval Bhogayata Jun 01 '22 at 12:48
I'm not following, why does the docker container need a kubernetes environment? I understand your kubernetes is running in Digitalocean, which essentially means there is a public URL exposed by Kubernetes that your host connects to using the configuration (URL and auth credentials) in the host's kube config file. Your program inside the container needs to be able to connect to that URL and authenticate so it needs to get it from somewhere, like, for example, the kube config file. – Gerard Garcia Jun 01 '22 at 12:53
You don't need a Kubernetes _per se_, just the client libraries (compiled into your Rust application) and the `$KUBECONFIG` file (the bind-mounted file). – David Maze Jun 01 '22 at 13:27
@GerardGarcia, I tried the solution as per ur suggestion. I added my docker-compose in the question, please do have a look and let me know if you have any inputs. – Keval Bhogayata Jun 02 '22 at 11:52
A bind mount will be easier to use to get a single file from the host inside the container. Sldo, it may work with a custom path and the $KUBECONFIG env variable but since I haven't seen if the rust Kubernetes library actually looks at this ENV, I prefer to just put it in the standard place. I've edited my answer to show how it will look like. – Gerard Garcia Jun 02 '22 at 12:08
@GerardGarcia, Also tried putting the config at the standard place - as per your updated answer, but no improvement ! – Keval Bhogayata Jun 02 '22 at 12:52
How are you initializing the configuration in your application? – Gerard Garcia Jun 02 '22 at 13:03
And does this command show the kube configuration? `docker exec -it \`docker ps -q -f=name=custom-tool\` cat /root/.kube/config` – Gerard Garcia Jun 02 '22 at 13:10

Allow docker app to access kubectl pointed cluster

1 Answers1