Secret can't be converted into key names and value pairs

Question

While creating a secret using python embedding in bash, I am getting 'AWS Secret can't be converted into key names and value pairs'. Firstly wrote all in bash using aws cli command where I was also getting this error then read in one of the articles to use json.dumps function of python which also doesn't work. Any ideas. Please see code and image below.

Note:

It work with short string "hello: world", but with a long string (key and /or csr) it gives aforementioned error.

Moreover, key values are there in a plain text but not in Key/value section is what an issue is.

cat << EOF > pyscript.py
#!/usr/bin/python3 -tt
import json
import boto3    

client = boto3.client('secretsmanager')
key_info = json.dumps([{"csr":"""$csr"""},{"config":"""$config"""},{"PrivateKey":"""$prvpem"""}])
response = client.create_secret(
    Name='marw',
    KmsKeyId='alias/SecretsMgr',
    SecretString=key_info
)
print(response)
EOF

chmod 770 pyscript.py
./pyscript.py

printf "This is BASH again\n"

Tried AWS cli as given below with the same issue. Key pairs are entered in plaintext but not in key/value section

export prvpem=`cat ${keyfile}`
export csr=`cat ${csrfile}`
export config=`cat ${key_algorithm}-$line.cfn`
# echo  -e "${BLU}${prvpem}${WHT}"
aws secretsmanager create-secret --name marwahas51 \
    --description "Values for this environments wildcard certifcate in ACM" \
    --secret-string "{\"Privatekey\":\"$prvpem\",\"csr\":\"$csr\",\"config\":\"$config\"}" \
    --kms-key-id "alias/SecretsMgr" \
   --tags "Key=Environment,Value=Dev"

Edit: Scenario 1 When string is --secret-string '{"Privatekey": "$prvpem"}' Output is:

Scenario 2

When string is

--secret-string "{\"Privatekey\":\"$prvpem\"}" \

Output is:

Could it be the quotes? Start with something simple and see if it works: `json.dumps([{"my_key":"my_value"}])` Frankly, calling Python from Shell is a bit strange -- it should really use the AWS CLI. Show us what you tried and what error you received and we can help you with using the AWS CLI. — John Rotenstein, Jun 03 '23 at 20:25
@JohnRotenstein it work with short string "hello: world", but with a long string (key) it gives error. See cli code in a next message. Also, note that key values are there in a plain text but not in Key/value section. — user3561229, Jun 03 '23 at 22:11
@JohnRotenstein - Added AWS CLI in original post + a few more details. — user3561229, Jun 03 '23 at 22:27
You can't have new lines in the middle of a JSON string. Use `\n` instead. Use a JSON linter with the plaintext view to see the issue more clearly. — kichik, Jun 04 '23 at 19:47
@kichik New lines will be part of the private key and csr being created as part of the process before storing in secrets manager. — user3561229, Jun 04 '23 at 22:31
OK. They are still invalid in JSON. You are generating invalid JSON. You need to use `\n`. — kichik, Jun 04 '23 at 22:45

score 0 · Answer 1 · answered Jun 03 '23 at 23:35

0

The boto3 documentation shows an example of how to provide the key/value pairs:

SecretString='{"username":"david","password":"EXAMPLE-PASSWORD"}',

Note that it is a single dictionary of values, not a list of dictionaries.

For the AWS CLI, try wrapping in single quotes with escape characters:

--secret-string '{"key": "value"}'

answered Jun 03 '23 at 23:35

John Rotenstein

241,921
22
380
470

It works only when it's a small string as mentioned in my original post. It work with short string "hello: world", but with a long string (key and /or csr) it gives aforementioned error. – user3561229 Jun 04 '23 at 04:15
Tried many times and played around with wrapping with same results where keys, csr and config files are entered in plaintext section, but for key/value section with an error 'Secret can't be converted into key names and value pairs' – user3561229 Jun 04 '23 at 04:17
--secret-string "{\"Privatekey\":\"$prvpem\",\"csr\":\"$csr\",\"config\":\"$config\"}" \ It prints the values, but only in plaintext section, but for key/value section with an error 'Secret can't be converted into key names – user3561229 Jun 04 '23 at 04:20
Did you try it without the backslashes as shown in my example? – John Rotenstein Jun 04 '23 at 06:41
It works, but doesn't print value of a variable. It also works when the value of a variable is a short string when I use escape characters, but when used on a long string like CSR or private key, it throws an error like mentioned. See new images in a original question. – user3561229 Jun 04 '23 at 18:55
[How do I handle newlines in JSON?](https://stackoverflow.com/a/42073/174777) suggests that newlines should be escaped. Thus, the newlines in the Private Key might be invalidating the JSON. – John Rotenstein Jun 05 '23 at 07:49

score 0 · Accepted Answer · answered Jun 06 '23 at 03:11

Thankyou @johnRotenstein and @kichik.

I tried to play around strings and escape characters, but nothing worked. When @kichik mentioned "You can't have new lines in the middle of a JSON string", then I thought of encoding key, csr and config file to base 64 which will be all alphanumeric characters, which finally worked. When I read any of the files back, I will decode a string.

# Encode private key, csr and confile file to base64 to save string as json in to ASM

base64_private_key=$(cat ${keyfile} |base64 -w0)
base64_csr=$(cat ${csrfile} |base64 -w0)
base64_config=$(echo ''${config1}'' |base64 -w0)

# echo "Command below on how to decode base64"
# echo ''${base64_private_key}'' |base64 --decode



# Create ASM and load base64 encoded private key, csr and confile file
aws secretsmanager create-secret --name <<name of a secret>> \
  --description "Values for this environments wildcard certifcate in ACM" \
  --secret-string "{\"EncBase64Privatekey\":\"$base64_private_key\",\"EncBase64CSR\":\"$base64_csr\",\"EncBase64Config\":\"$base64_config\"}" \
  --kms-key-id "alias/SecretsMgr" \
  --tags "Key=Environment,Value=Dev"

Secret can't be converted into key names and value pairs

2 Answers2