Parsing secrets from AWS secrets manager using AWS cli

Question

I am retrieving secrets I have stored in AWS secrets manager with the AWS cli like this:

aws secretsmanager get-secret-value --secret-id secrets

Which returns

arn:aws:secretsmanager<ID>:secret:my_secrets <number> my_secrets {"API_KEY":"ABCDEFGHI"}       <UUID string>
VERSIONSTAGES   AWSCURRENT

Does anyone know how I only get the secret ("API_KEY": "ABCDEFGHI")? I need to move these secrets to my register-task-definition environment variables. The best way would be to store them in a file and delete it after us or store them in variable. It is running on a linux machine.

helloV · Accepted Answer · 2019-12-25T15:37:24.277

129

Use the --query option of the CLI to extract just the secret.

aws secretsmanager get-secret-value --secret-id secrets --query SecretString --output text

edited Dec 25 '19 at 15:37

answered Jun 18 '18 at 19:10

helloV

50,176
7
137
145

2

Is this no longer working? every secret is null for me – Cale Oct 03 '18 at 00:14
5

Ah `SecretString` selectes the entire secret string, it is not used to match a secret key – Cale Oct 03 '18 at 00:16
@caleb Just running the command resulted in that error or w -query did it work? – committedandroider Jul 27 '19 at 00:27
Came to a wrong place but found an answer (`--output text`) :) – dance2die Oct 27 '22 at 21:16

score 36 · Answer 2 · answered Oct 22 '18 at 00:58

36

aws secretsmanager get-secret-value --secret-id secrets| jq --raw-output '.SecretString' | jq -r .API_KEY

using jq you can print.

answered Oct 22 '18 at 00:58

Vijay Kumar

479
4
4

17

jq is not a standard util – Balaji Boggaram Ramanarayan May 04 '19 at 00:20
4

And if your secret has a dash, you can use: `aws secretsmanager get-secret-value --secret-id my-secret --query SecretString --output text | jq '."my-secret"'` – Yann Stoneman Feb 21 '21 at 14:49
2

Correction, to remove quotes in addition to making it work for a secret with a dash: `aws secretsmanager get-secret-value --secret-id my-secret --query SecretString --output text | jq -r '."my-secret"'` – Yann Stoneman Feb 21 '21 at 15:27

kadir · Answer 3 · 2019-12-16T20:42:45.660

30

Small addition to helloV answer. You can add the output parameter text to remove the quotes.

aws secretsmanager get-secret-value \
   --secret-id secrets \
   --query SecretString \
   --output text

edited Dec 16 '19 at 20:42

answered May 06 '19 at 23:24

kadir

1,417
11
35

score 12 · Answer 4 · answered Oct 31 '19 at 19:38

12

When you have multiple secret and you get json return, you can use get the exact value of password by using

aws secretsmanager get-secret-value --secret-id <secret_bucket_name> | jq --raw-output '.SecretString' | jq -r .key_for_password

answered Oct 31 '19 at 19:38

Aseem Jain

333
2
7

jq is json utility for shell command. it helps to parse the Json and pull the attribute from the file. – Aseem Jain Apr 14 '20 at 18:26
2

this is a copy-paste of this answer https://stackoverflow.com/a/52921217/4212158 – crypdick Aug 31 '20 at 16:33
In my case, The password has '&' symbol and is converted to '\u0026'. Is there any workaround for this? – Auro Sarma Apr 21 '21 at 19:24

Jon · Answer 5 · 2023-08-24T04:41:42.073

12

If your secret will only have one key/pair value, and you only want the value to be printed out, and you don't want to rely on your system having jq installed first, you can do:

aws secretsmanager get-secret-value --secret-id secrets --query SecretString --output text | cut -d: -f2 | tr -d \"}

edited Aug 24 '23 at 04:41

answered Feb 14 '21 at 04:57

Jon

952
1
11
17

you're missing a `"\"` before the last `}`, like `tr -d \"\}` – GwenM Mar 09 '21 at 15:19
@GwenM not needed on GNU/Bash – Jon Mar 09 '21 at 21:01
ah yeah you might be right, I was using ZSH on mac – GwenM Mar 10 '21 at 11:08
1

a good answer which doesn't require jq – Pierre Nov 01 '21 at 20:55

score 11 · Answer 6 · answered Oct 31 '19 at 13:11

11

So I faced a bit of trouble in extracting what I needed, the value for my two variables that I stored in SecretsManager. So here is what worked for me.

NOTE: It's an example from the AWS SecretsManager doc.

I ran this

aws secretsmanager get-secret-value --secret-id MyTestDatabaseSecret --version-stage AWSPREVIOUS

The response of this query is:

{
  "ARN": "arn:aws:secretsmanager:us-west-2:123456789012:secret:MyTestDatabaseSecret-a1b2c3",
  "Name": "MyTestDatabaseSecret",
  "VersionId": "EXAMPLE1-90ab-cdef-fedc-ba987EXAMPLE",
  "SecretString": "{\n  \"username\":\"david\",\n  \"password\":\"BnQw&XDWgaEeT9XGTT29\"\n}\n",
  "VersionStages": [
    "AWSPREVIOUS"
  ],
  "CreatedDate": 1523477145.713
}

Now I want to get the value of username or password to be precise

aws secretsmanager get-secret-value --secret-id MyTestDatabaseSecret --version-stage AWSPREVIOUS | jq --raw-output .SecretString | jq -r ."password"

Output

BnQw&XDWgaEeT9XGTT29

answered Oct 31 '19 at 13:11

thecloudguy

331
3
8

I want to list all secrets and then get all values and run grep against it – Ashish Karpe Oct 20 '21 at 02:38
> aws secretsmanager get-secret-value --secret-id * is not working for me – Ashish Karpe Oct 20 '21 at 02:39
1

what is the error message? Do you have access rights to all? Are you passing the version number in your query? Instead of using * wild character, can you make a list of secret-id and then try to run a loop. First, try to run it for 10 odd secrets and then check to scale it up. – thecloudguy Oct 22 '21 at 17:45
1

https://github.com/ashishkarpe/scripts_aws_cli/blob/main/find_secrets_values.sh have written script which worked for me thanks – Ashish Karpe Oct 27 '21 at 03:57

Heath Borders · Answer 7 · 2022-09-19T23:17:29.350

7

Lots of answers here depend on jq. If you don't want to install any other dependencies, you can use a python3 one-liner:

aws secretsmanager get-secret-value \
   --output text \
   --query SecretString \
   --secret-id my-secret-name \
| python3 -c 'import json, sys; print(json.load(sys.stdin)["my-secret-key"])'

Based on helloV's answer.

edited Sep 19 '22 at 23:17

answered Oct 20 '21 at 21:48

Heath Borders

30,998
16
147
256

Thank you for this. You can also assign this to a variable this way: `SECRET_ACCESS_KEY=$(aws secretsmanager get-secret-value \ --output text \ --query SecretString \ --secret-id my-secret-name \ | python3 -c 'import json, sys; print(json.load(sys.stdin)["my-secret-key"])')` – Promise Preston Apr 23 '23 at 08:22

score 6 · Answer 8 · answered Aug 17 '22 at 15:50

6

All answers working but require 3rd party integration ( mainly jq ). the following bash command grabs the relevant Value without any other 3rd party solution -

SECRET_ARN=arn:aws:secretsmanager:eu-west-1:123456:secret:/test
SECRET_KEY=DB_PASSWORD
aws secretsmanager get-secret-value \ 
  --secret-id $SECRET_ARN \ 
  --query SecretString \
  --output text | grep -o '"$SECRET_KEY":"[^"]*' |  grep -o '[^"]*$'

answered Aug 17 '22 at 15:50

Amit Baranes

7,398
2
31
53

The answered marked as the correct one does not require jq: aws secretsmanager get-secret-value --secret-id secrets --query SecretString --output text – Moddaman Aug 18 '22 at 09:03
4

The marked answer gets all secret values. my example grabs a specific value based on a secret key. – Amit Baranes Aug 19 '22 at 11:11

score 1 · Answer 9 · answered Jun 30 '21 at 12:14

PowerShell solution without Jq

$a = aws secretsmanager get-secret-value --region <region> --secret-id <secret-name>  | ConvertFrom-Json

$a all json converted to objects type

Output

ARN           : xxxxxx
Name          : postgxxx
VersionId     : fxxxx-xx-x-xx
SecretString  : {"key":"value","key2":"value"}
VersionStages : {xxxxx}
CreatedDate   : xxxxx.xx

$b = $a.SecretString | ConvertFrom-Json

Output

key : value
key2 : value

$b.key

**Output** 
value

score 1 · Answer 10 · answered Oct 27 '21 at 03:59

1

Script to List all available AWS secrets to a /tmp/name.text and find specific secret values from it

Note needs AWS CLI configure to run this script successfully

#!/bin/bash

aws secretsmanager list-secrets | grep  "Name" | awk '{print $2}' | tr -d '"' | sed 's/,/ /g' > /tmp/name.text

for line in `cat /tmp/name.text`
do

echo $line >> /tmp/secrets-values.txt

aws secretsmanager get-secret-value --secret-id "$line" | grep "XYZ" >>  /tmp/secrets-values.txt
done

answered Oct 27 '21 at 03:59

Ashish Karpe

5,087
7
41
66

1

Did it work? You tested. If yes, then great. :) – thecloudguy Oct 27 '21 at 05:39
yes I tested it and it worked – Ashish Karpe Oct 27 '21 at 05:41

score 1 · Answer 11 · answered Nov 03 '21 at 16:01

In the vein of "... without jq" answers, here's one for node users. (requires modern bash and nodejs, could easily be rewritten to just use sh by using an echo | instead of the cleaner <<<)

SECRET_ARN="..."
REGION=us-east-1

SECRET_BLOB=$(aws secretsmanager get-secret-value --region="$REGION" --output=text --query SecretString --secret-id "$SECRET_ARN")

MY_VALUE=$(node -pe 'JSON.parse(require("fs").readFileSync("/dev/stdin").toString()).myKey' <<< "$SECRET_BLOB")

MY_OTHER_VALUE=$(node -pe 'JSON.parse(require("fs").readFileSync("/dev/stdin").toString()).myOtherKey' <<< "$SECRET_BLOB")

If you need to pull multiple values from the secret, you'll want to cache the json blob in an env var. If you only need a single value though:

MY_VALUE=$(aws secretsmanager get-secret-value --region="$REGION" --output=text --query SecretString --secret-id "$SECRET_ARN" | node -pe 'JSON.parse(require("fs").readFileSync("/dev/stdin").toString()).myKey' <<< "$SECRET_BLOB")

score 1 · Answer 12 · answered Sep 30 '22 at 01:28

I see many JQ examples but Powershell has a pretty awesome integration with AWS. This is the way I do it in Powershell:

Your JSON value

{"API_KEY":"ABCDEFGHI"}

$aws_secret = Get-SECSecretValue -SecretId my_secrets
$mysecret = $aws_secret.SecretString | ConvertFrom-Json
$myapikey = $mysecret.API_KEY
$newsecret = ConvertTo-SecureString -String $myapikey -AsPlainText -Force

The value from the secret manager is a JSON which Powershell can natively convert into a type of array that you can reference. I convert it back into a secure string under the assumption its a secret and you want to pass it in. The code above should work for you. Let me know if you run into any issues with the code I provided.

score -3 · Answer 13 · answered Jun 23 '20 at 20:55

Use this to get just the value of the secret key. Make sure to fill in your secert ID and the key of the secret:

aws secretsmanager get-secret-value --secret-id <yourSecretID> | jq '.SecretString' | tail -c +2 | head -c -2 | tr -d '\' | jq .<YourSecretKey>