17

I want to update my deployment on kubernetes with a new image which exists on 'eu.gcr.io' (same project), I have done this before. But now the pods fail to pull the image because they are not authorized to do so. This is the error that we get in the pod logs.

Failed to pull image "eu.gcr.io/my-gcp-project/my-image:v1.009": 
rpc error: code = Unknown desc = Error response from daemon: 
unauthorized: You don't have the needed permissions to perform this operation,
and you may have invalid credentials.

The service account on the cluster has kubernetes admin and storage admin roles which should be sufficient. But even when I make the service account project editor (for debugging sake) it still doesn't work (same error).

I have also tried creating a fresh new cluster (default settings) and apply my deployment there, but then I got the exact same issue.

I'm not sure what I can try anymore.

Any help or suggestions are greatly appreciated.

EDIT:

I just found out that I can still pull and deploy older images. But every new image I build cannot be pulled by the kubernetes pods.

howie
  • 2,587
  • 3
  • 27
  • 43
Georges Lorré
  • 443
  • 3
  • 11

3 Answers3

8

According to your desciption

I just found out that I can still pull and deploy older images. But every new image I build cannot be pulled by the kubernetes pods.

I assume you can pull docker image by command, but not kubectl.

docker pull eu.gcr.io/my-gcp-project/my-image:v1.009

So reference by this article Using Google Container Registry with Kubernetes, the authenication is differnet between pull docker image by docker pull and kubectl .

Did you give access token to GKE?

kubectl create secret docker-registry gcr-access-token \
--docker-server=eu.gcr.io \
--docker-username=oauth2accesstoken \
--docker-password="$(gcloud auth print-access-token)" \
--docker-email=any@valid.email
howie
  • 2,587
  • 3
  • 27
  • 43
  • Yes this did it ! I had to create an access token. This article is a great find thank you howie. It is quite strange that this is all of the sudden necessary since it should work out of the box. – Georges Lorré Mar 05 '19 at 09:19
3

You will need to create a docker-registry secret and use imagePullSecrets in you pod definition:

kubectl create secret docker-registry regcred --docker-server=<your-registry-server> --docker-username=<your-name> --docker-password=<your-pword> --docker-email=<your-email>


apiVersion: v1
kind: Pod
metadata:
  name: private-reg
spec:
  containers:
  - name: private-reg-container
    image: <your-private-image>
  imagePullSecrets:
  - name: regcred

see this guide for more information

Amityo
  • 5,635
  • 4
  • 22
  • 29
  • 1
    This should not be needed since I'm pulling the image from google container registry (in the same gcp project) and not from docker. This has worked before. But now the k8s pods are unauthorized. – Georges Lorré Mar 03 '19 at 09:51
0

The service account or user used in k8s cluster don't have access to gcr (google container registry). To allow access there are two methods, so that k8s node can pull the image from gcr and deploy the pods.

Method 1 : Make the Container registry Public (which is not recommended) only for lab or test purpose.

Method 2 : Add Storage object admin or storage object viewer role to service account assigned to your k8s cluster and nodes. (recommended)

chinmay mandal
  • 1
  • 1