4

I am trying to pull images from my ecr repository as well as from dockerhub using Nomad. The problem is that if I don't pull those images myself, Nomad won't pull them and will complain with the error:

Failed to find docker auth for repo "envoyproxy/envoy": docker-credential-ecr-login with input "envoyproxy/envoy" failed with stderr: exit status 1

It will easily pull the ECR images, but those images required for sidecars or non-ecr images deployed by me, for example postgres, won't be pulled with same error. Did anybody else encounter same issue?

Alan Sereb
  • 2,358
  • 2
  • 17
  • 31

3 Answers3

2

I had this same issue, I'm not sure if there's a way around it if you're just using this stanza:

plugin "docker" {
  config {
    auth {
      helper = "ecr-login"
    }
  }
}

Alternatively, I set this:

plugin "docker" {
  config {
    auth {
      config = "/opt/docker.json"
    }
  }
}

And then populated the file at /opt/docker.json with the following values:

{
  "credHelpers": {
    "000000000000.dkr.ecr.us-west-2.amazonaws.com": "ecr-login"
  },
  "auths": {
    "https://index.docker.io/v1/": {}
  }
}

Replace 000000000000 with your aws account id and us-west-2 with your region.

maxm
  • 3,412
  • 1
  • 19
  • 27
  • This is a nice workaround, I haven't thought of this, but where do you put the plugin stanza? The point is that I do not want to be changing it in the agent configuration all the time.. – Alan Sereb Dec 07 '20 at 18:39
  • That plugin stanza is in my agent configuration, this section here: https://www.nomadproject.io/docs/configuration/plugin. You would add it to an agent client configuration like this: https://github.com/hashicorp/nomad/blob/master/terraform/shared/config/nomad_client.hcl – maxm Dec 07 '20 at 19:26
  • Then I am a little bit confused, how am I supposed to download both: ecr and non-ecr images? – Alan Sereb Dec 08 '20 at 14:21
  • both will work with the config I have supplied. When requesting ecr images your images will start with the hostname that you fill into the config, something like: `000000000000.dkr.ecr.us-west-2.amazonaws.com` and they will be matched to the ecr-login auth method. When you pull regular docker images they will be requested from the docker registry at `https://index.docker.io/v1/`. This configuration works for me personally for both my ECR images and my docker hub images. – maxm Dec 08 '20 at 17:36
  • 1
    Thanks, you're a lifesaver! – Robert Hafner Dec 02 '21 at 18:27
0

For mixed ecr and non-ecr workload in recent nomad version it seems totally fine to point the auth config to a docker configuration that only includes the credHelpers stanza:

/etc/nomad.d/nomad-docker.hcl

plugin "docker" {
  config {
    auth {
      config = "/root/.docker/config.json"
    }
  }
}

/root/.docker/config.json

{
    "credHelpers": {
        "**********.dkr.ecr.eu-central-1.amazonaws.com": "ecr-login"
    }
}
Gunnar Thielebein
  • 1
  • 1
0

The workaround I have found was adding:

"auth_soft_fail": true

Add this as part the config.

Nomad cannot pull other images if credential helper is in place

minTwin
  • 1,181
  • 2
  • 21
  • 35
Anderson Carvalho
  • 1