Access custom docker registry with NodePort service

Question

I have a local kubernetes cluster up and running using k3s. It works like a charm so far.

On it I'm running a custom Docker registry from which I want to pull images for other deployments.

The registry is exposed to the host by means of a NodePort service. Internally it has port 5000, externally it's on port 31320.

I can push docker images to the registry from the host by tagging them as myhostname:31320/myimage:latest. This works great too.

Now I want to use this image in a basic Job deployment. I'm using the whole tag myhostname:31320/myimage:latest as container image entry like this:

apiVersion: batch/v1
kind: Job
metadata:
  name: hello-world
spec:
  template:
    metadata:
      name: hello-world-pod
    spec:
      containers:
        - name: hello-world
          image: myhostname:31320/myimage:latest
      restartPolicy: Never

Unfortunately, I keep getting a 400 BadRequest error stating: image can't be pulled. If I try using the internal service name of the registry and the internal port instead, like in private-registry:5000/myimage:latest, I'm getting the same error.

I suppose I cannot use private-registry:5000/myimage:latest because that's just not the tag of the image. I cannot push the image to private-registry:5000/myimage:latest because the host private-registry is only known inside the cluster and the port 5000 is not exposed to the host.

So... I'm stuck. What am I going to do about this? How do I get to push images from the host to the registry and allow them to be pulled from inside the cluster?

Answer 1

Kubernetes has a rich documentation on how to implement multiple registries to allow further deployments/pods to access to public or even private registries, to do so you can create an image pull secret k8s ressource (docs), you can either create it by running this command:

kubectl create secret docker-registry regcred --docker-server=<your-registry-server> --docker-username=<your-name> --docker-password=<your-pword>

or by deploying this resource in your cluster:

apiVersion: v1
kind: Secret
metadata:
  name: myregistrykey
  namespace: awesomeapps
data:
  # Make sure the you convert the whole file to base64!
  # cat registry.json | base64 -d
  .dockerconfigjson: <registry.json>
type: kubernetes.io/dockerconfigjson

registry.json example

{
    "auths": {
        "your.private.registry.example.com": {
            "username": "janedoe",
            "password": "xxxxxxxxxxx",
            "email": "jdoe@example.com",
            "auth": "c3R...zE2"
        }
    }
}

And now you can simply attache this imagePullSecret resource you can attache it to your deployment:

apiVersion: batch/v1
kind: Job
metadata:
  name: hello-world
spec:
  template:
    metadata:
      name: hello-world-pod
    spec:
      imagePullSecrets:
        - name: regcred
      containers:
        - name: hello-world
          image: myhostname:31320/myimage:latest
      restartPolicy: Never  

PS

You might also consider adding your registry in docker daemon as insecure registry if you encounter other issues.

you can check this SO question