aws ecs fargate can't fetch secret manager

Question

I'm using AWS ECS service for orchestrate my docker container.

Also used Secret Manager for stored and retrieve personal information.

I apply SecretsManagerReadWrite policy to my ecsTaskExecutionRole and ecsServiceRole.

Before using Fargate, I just used ECS with EC2.

And it works fine.

But in fargate, it throw NoCredentialsError

I fetched to secret manager via python script that made with boto3. (https://docs.aws.amazon.com/ko_kr/code-samples/latest/catalog/python-secretsmanager-secrets_manager.py.html)

Is there any solution here?

Thanks.

CUSTOM Permission

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "kms:Decrypt",
                "secretsmanager:GetSecretValue",
                "ssm:GetParameters"
            ],
            "Resource": "*"
        }
    ]
}

Sébastien Stormacq · Accepted Answer · 2019-04-08T07:54:02.857

4

Be sure that the IAM policy you applied has the following permissions :

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ssm:GetParameters",
        "secretsmanager:GetSecretValue",
        "kms:Decrypt"
      ],
      "Resource": [
        "arn:aws:ssm:<region>:<aws_account_id>:parameter/parameter_name",
        "arn:aws:secretsmanager:<region>:<aws_account_id>:secret:secret_name",
        "arn:aws:kms:<region>:<aws_account_id>:key/key_id"
      ]
    }
  ]
}

Also, be sure that you are using Fargate 1.3.0 (https://docs.aws.amazon.com/AmazonECS/latest/developerguide/platform_versions.html)

But I would try something else to reduce the amount of code. Since Nov 2018, it is not necessary to write your own code to fetch secrets from Secret Manager. ECS/Fargate can do it for you. Just give ECS the permission to access your secret and give the secret ARN in the task definition. ECS/Fargate will assign the secret to the environment variable. Your code just need to read the environment variable as usual.

For example :

"containerDefinitions": [
    {
        "secrets": [
            {
                "name": "environment_variable_name",
                "valueFrom": "arn:aws:ssm:region:aws_account_id:parameter/parameter_name"
            }
        ]
    }
]

Doc is here : https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data.html

edited Apr 08 '19 at 07:54

answered Apr 08 '19 at 07:47

Sébastien Stormacq

14,301
5
41
64

You mean that I just assign arn to my task definition, it stored into environment variable automatically? Or I have to additional behavior for it? I want to stored all of my secret manager's variable to my system's environment variable. Because parsed it in my application. – Hide Apr 08 '19 at 07:50
correct, you can assign a secret ARN to an environment variable in your Task Definition. At code level, you just need to read the env. variable, like usual. Under the scene, ECS/Fargate will fetch the secret from Secret Manager. This reduces the amount of AWS specific code to write into your app. – Sébastien Stormacq Apr 08 '19 at 07:52
I created custom policy(your first code) in `iam - policy` and attached it to `ecsTaskExecutionRole` and `ecsServiceRole`, but it doens't work too. Is it wrong way? – Hide Apr 08 '19 at 08:15
Also added my custom permission json to my post. – Hide Apr 08 '19 at 08:16
`ecsTaskExecutionRole` should be enough. These are the permission the container has access to at runtime. Just tested on a cluster here and works – Sébastien Stormacq Apr 08 '19 at 16:59
I think your suggestion is related ssm `parameter store`. But I using `Secret Manager`. – Hide Apr 09 '19 at 01:02
Now, my proposition is both working with parameter store and secrets manager. Did you correctly use the key `valueFrom` (and not `value`) in your task definition's JSON ? – Sébastien Stormacq Apr 09 '19 at 06:28
Also, be sure to use the key `secret` and nnot store your ARN in the `environment` section. Exemple from my task def : ```"secrets": [ { "valueFrom": "arn:aws:secretsmanager:eu-west-1:48000000093:secret:my_secret-SUr3hB", "name": "SECRET" }``` – Sébastien Stormacq Apr 09 '19 at 06:30
I use `valueFrom` correctly. Anyway, I changed `SecretManager` to `SSM`, and it solved. Thanks :) – Hide Apr 09 '19 at 06:38
Maybe helpful to someone: Be sure to select "ssm:GetParameters" and not "ssm:GetParameter" when creating the policy using the visual editor. – David Nov 05 '20 at 10:04

score 0 · Answer 2 · edited Oct 18 '21 at 21:48

I stumbled upon this thread while troubleshooting the same issue. In my case the permissions were properly configured. However, the ARN of the Secrets Manager was not complete.

I had passed the ARN as:

arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:nonprod-testapp-rds-password"

Instead of:

arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:nonprod-testapp-rds-password-wdxsae

The issue got resolved after passing the complete ARN of the secret as Secrets in container definition

aws ecs fargate can't fetch secret manager

2 Answers2