6

I am using docker containers with secrets on ECS, without problems. After moving to fargate and platform 1.4 for efs support i start getting the following error.

Any help please?

ResourceInitializationError: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve secret from asm: service call has been retried 1 time(s): secret arn:aws:secretsmanager:eu-central-1:.....

Rob Daalman
  • 61
  • 1
  • 2

2 Answers2

4

Here's a checklist:

  1. If your ECS tasks are in a public subnet (0.0.0.0/0 routes to Internet Gateway) make sure your tasks can call the "public" endpoint for Secrets Manager. Basically, outbound TCP/443.
  2. If your ECS tasks are in a private subnet, make sure that one of the following is true: (a) your instances need to connect to the Internet through a NAT gateway (0.0.0.0/0 routes to NAT gateway) or (b) you have an AWS PrivateLink endpoint to secrets manager connected to your VPC (and to your subnets)
  3. If you have an AWS PrivateLink connection, make sure the associated Security Group has inbound access from the security groups linked to your ECS tasks.
  4. Make sure you have set GetSecretValue IAM permission to the ARN(s) of the secrets manager entry(or entries) set in the ECS "tasks role".

Edit: Here's another excellent answer - https://stackoverflow.com/a/66802973

eternaltyro
  • 336
  • 2
  • 12
  • Thank you. The first point is what solved it for me. It wasn't really obvious ECS tries to reach to Secrets Manager so that makes sense. – squeekyDave Apr 04 '23 at 20:44
  • THANK YOU for pointing me in the right direction! I had an ECS task in a public subnet, with a sec group configured to allow ALL egress traffic, and still could not pull secrets from ASM/SSM. Turned out the fix was to allow INGRESS traffic on 443 to the ECS service/task, for...some reason. – EvanK Apr 10 '23 at 23:46
0

I had the same error message, but the checklist above misses the cause of my problem. If you are using VPC endpoints to access AWS services (ie, secretsmanager, ecr, SQS, etc) then those endpoints MUST permit access to the security group that is associated with the VPC subnet that your ECS instance is running in.

Another watchit is, if you are using EFS to host volumes, ensure that your volumes can be mounted by the same security group identified above. Go to EFS, select the appropriate file system, Network tab, then Manage.

Dharman
  • 30,962
  • 25
  • 85
  • 135
Andrew Selby
  • 336
  • 2
  • 6