Kubernetes cannot pull image from private docker image repository

Question

I have problem with kubernetes (minikube) and pull images from local image repository on docker. Docker repository was created:

docker run --entrypoint htpasswd registry:2 -Bbn zordon examplePassword > /mnt/LINUX/auth/htpasswd

docker run -d \
  -p 5000:5000 \
  --restart=always \
  --name registry \
  -v /mnt/LINUX/dockerreg:/var/lib/registry \
  -v /mnt/LINUX/auth:/auth \
  -e "REGISTRY_AUTH=htpasswd" \
  -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \
  -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
  registry:2

Then I want to create simple pod with image which was succesfully uploaded to local repository:

curl localhost:5000/v2/_catalog
{"repositories":["car/configuration"]}

I have also create secret on minikube cluster with:

kubectl create secret docker-registry docregkey --docker-server=localhost:5000 --docker-username=zordon --docker-password=examplePassword --docker-email=test@dock.mail

and define simple Pod:

    apiVersion: v1
kind: Pod
metadata:
  name: private-reg
spec:
  containers:
  - name: private-reg-container
    image: car/configuration:latest
    imagePullPolicy: Always
  restartPolicy: Always
  imagePullSecrets:
  - name: docregkey

unfortunatelly I getting still:

Failed to pull image "car/configuration:latest": rpc error: code = Unknown desc = Error response from daemon: pull access denied for car/configuration, repository does not exist or may require 'docker login'

How i can fix this problem ?

this question is completely and correctly answered in the official documentation, here is a deep link to it: https://kubernetes.io/docs/tutorials/hello-minikube/#create-a-docker-container-image — VeRo, Oct 25 '18 at 22:24
@vero While that is no doubt an excellent tutorial, it doesn't solve the question — conjectures, Apr 23 '21 at 18:25

score 22 · Answer 1 · answered Feb 14 '20 at 17:46

For minikube to pull from your own local docker registry, the tag affects the pull policy. Per Images docs, pull policy is IfNotPresent by default EXCEPT if

you use :latest as the tag for the image to use
OR you omit the tag for the image to use.

In those cases the pull policy will effectively default to Always, which will attempt to pull from docker hub. This will cause minikube to be unable to fetch local images that have no tag or "latest" tag.

Moral of the story is, don't rely on the default because it is too confusing :)

So always explicitly state the pull policy:

when deploying into minikube the pull policy should be IfNotPresent or Never for the local images
when deploying into a cloud host (like AWS), pull policy should be as for public images (see below)
the pull policy should be Always for those public images that use a tag like "latest" or "stable" (because the image the tag points will change over time), and IfNotPresent for tags that always point to the same image (to avoid fetching more than necessary)

This means that if you avoid using tags like latest and stable etc, there is only one rule to follow:

explicitly set the imagePullPolicy in your spec (or on the command line in the case of run) to IfNotPresent, as this is will always look for it locally first, and go to public registry if it is not found locally, and this will work whether or not you are deploying into minikube or cloud.

Thank you Oliver, this got me going with minikube, docker, and a local image. `docker run image_name --image-pull-policy=IfNotPresent` — Paul Watson, Aug 08 '20 at 14:12

score 19 · Answer 2 · edited Jul 05 '19 at 11:32

The problem is with the image pull policy - you have set this to Always (the default setting). This means that the Docker deamon always tries to pull the image from the outer Docker registry - you want to use the local one instead.

Try to add --image-pull-policy=Never when creating a deployment.

A good tutorial of using locally created images is here (it helped me):

https://kubernetes.io/docs/tutorials/hello-minikube/#create-a-docker-container-image

onuryartasi · Answer 3 · 2018-04-04T09:14:40.423

14

Because Minikube is VM not a your localhost. You try this code eval $(minikube docker-env) https://kubernetes.io/docs/getting-started-guides/minikube/

Open Terminal
eval $(minikube docker-env)
docker build .
kubectl create -f deployment.yaml

just valid this terminal. if closed terminal again open terminal and write eval $(minikube docker-env)

eval $(minikube docker-env) this code build image in Minikube

edited Apr 04 '18 at 09:14

answered Apr 04 '18 at 08:41

onuryartasi

587
2
6
13

what if docker image is build by maven plugin and jenkins ? – lukisp Apr 04 '18 at 15:06
Two way possible, First way you registry running in Minikube Second way remote private registry – onuryartasi Apr 04 '18 at 19:43
Thanks lot, works, after hrs of debugging! – Jose Mhlanga Jul 07 '21 at 13:10

Ognyan Dimitrov · Answer 4 · 2021-08-04T05:56:09.643

12

The same issue with me was that when I build an image with the local Docker which then is not added to minikube image list.

The resolution was to add the image manually:

minikube image load image-name:tag

edited Aug 04 '21 at 05:56

answered Jun 10 '21 at 08:20

Ognyan Dimitrov

6,026
1
48
70

It works for me, but still inconvenient when I need to rebuild image frequently when debugging. – nguyenhoai890 Sep 25 '21 at 06:04

score 7 · Answer 5 · answered Dec 28 '18 at 22:35

I wanted a one line solution to execute in my terminal. Everything else I tried was overly complex to auth ecr with minikube.

This is my command for aws ecr login that I run each day because the token expires. The examples below are for Debian 9 with AWS ECR.

shell

kubectl create secret docker-registry aws-ecr-credentials \
--docker-server=$ECR_REGISTRY \
--docker-username=AWS \
--docker-password=$(aws ecr get-login | awk '{print $6}') \
--docker-email=$IAM_EMAIL \
--namespace=$KUBE_NAMESPACE

template.yml

spec:
  imagePullSecrets:
    - name: aws-ecr-credentials

score 6 · Answer 6 · edited Mar 15 '19 at 19:15

6

The Problem is with the image name you are mentioning in the POD yaml file.

image: car/configuration:latest

This will try to pull from the global registry rather than local registry.Change the image name to include the repository too.

image: localhost:5000/car/configuration:latest

And make sure that you have included insecure registry in your docker daemon configuration if your registry is not secured.

edited Mar 15 '19 at 19:15

kenorb

155,785
88
678
743

answered Apr 04 '18 at 03:03

Pramod V

126
4

not work for me. The same eval $(minikube docker-env) – lukisp Apr 04 '18 at 15:06
Is the registry running on same node as minikube? – Pramod V Apr 04 '18 at 15:29
both are runing on same host (and also PC). docker image is buiild by maven plugin, and I see image in _catalog – lukisp Apr 04 '18 at 15:44
2

moreover in describe of pod I\m getting: `Failed to pull image "localhost:5000/car/configuration:latest": rpc error: code = Unknown desc = Error response from daemon: Get http://localhost:5000/v2/: dial tcp 127.0.0.1:5000: getsockopt: connection refused` but when try curl (`curl --user zordon:4a5b-9cd7 http://localhost:5000/v2/`) geting {} valid respone – lukisp Apr 04 '18 at 15:55
Ok one thing i could suspect from these errors is ideally when you run minikube it will be running on virtualized host.. so it will be trying to contact that localhost.. You can try exposing the registry using the host ip and use that host ip instead of the localhost for kubernetes. – Pramod V Apr 04 '18 at 16:27

ypaseka · Answer 7 · 2019-04-16T11:03:17.463

When you run Kubernetes in Docker for Desktop your applications will share the same image registry across Docker and Kubernetes. List od all images:

docker images --all

Choose of them and run it with changed atribute --image-pull-policy=Never. For example:

kubectl run ContainerName --image=myimage/server --port=8080 --image-pull-policy=Never

By default, the kubelet will try to pull each image from the specified registry. However, if the imagePullPolicy property of the container is set to IfNotPresent or Never, then a local image is used (preferentially or exclusively, respectively). Link

It's mean, that Kubernetes pull image from local registry, not remote cloud.

onuryartasi · Answer 8 · 2018-04-06T22:37:24.370

Private registry in Minikube

kubectl create -f kube-registry.yaml

(Grab kube-registry.yaml from this gist on github.)

and you need port-forward minikube to localhost (Just image build time)

kubectl port-forward --namespace kube-system \
$(kubectl get po -n kube-system | grep kube-registry-v0 | \
awk '{print $1;}') 5000:5000

After this, from the host curl localhost:5000 should return a valid response from the docker registry running on minikube

Repo : http://localhost:5000/v2/_catalog

Pull image : localhost:5000/image_name:image_tag

Reference: https://blog.hasura.io/sharing-a-local-registry-for-minikube-37c7240d0615

score 0 · Answer 9 · answered Dec 25 '21 at 22:57

The best approach to use minikube with a local registry i found was here:

Minikube Registry Handbook

Basically you will add a registry addon to minikube and do some port forward and redirects to access it both, inside kubernetes cluster, and from your host machine.

I tried one more thing, reference registry inside cluster using:

registry.kube-system.svc.cluster.local

Like this:

image: registry.kube-system.svc.cluster.local/postgres:latest

But it didnt work.

To work i got the cluster ip to the registry service:

kubectl get service --namespace kube-system

And referenced like this:

image: 10.99.112.119/postgres:latest

score 0 · Answer 10 · answered Apr 11 '22 at 19:17

Try to login docker on all the nodes (if its a cluster environment - docker login on the Master and the worker node).

[root@****]# docker login

Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. Username: Password:

Login succeed!

Then try to manually load the images on all the nodes

docker pull url

Kubernetes cannot pull image from private docker image repository

10 Answers10

Linked