Connect to Postgres database using AWS Secrets Manager

Question

Wanted to use AWS Secrets manager to login to postgres without using username and password as a plain text. i am not sure if this is doable, please forgive me if not. Currently this is what i am using to login to postgres using psycopg2:

 import psycopg2

conn = psycopg2.connect(host="hostname",port='5432',database="db", user="admin", password="12345")

i've already stored the username and password in secrets manager but not sure how to use it here. Please help

Parimal · Answer 1 · 2019-05-01T22:23:12.543

You can store your credentials (username/password) in SecretsManager using the console. You can store them as key value pairs, for example -

{ "username": "admin", "password": "12345" }

To use this in your Python script, you could do something like this -

session = boto3.session.Session()
client = session.client(
    service_name='secretsmanager',
    region=< region_name >
)
secret = client.get_secret_value(
         SecretId=secret_name
)
secret_dict = json.loads(secret['SecretString'])

username = secret_dict['username']
passw = secret_dict['password']

conn = psycopg2.connect(host="hostname",port='5432',database="db", user=username, password=passw)

Note that this is a simplified example without error handling. You also need to fill in the right region in place of < region_name > in the example.

The boto3 library may have changed since this answer in 2019; the region argument is now `region_name`. e.g. `session.client(service_name='', region_name='')` — developerjack, Jun 06 '21 at 15:06

score 0 · Answer 2 · answered May 01 '19 at 08:52

You should use the below process:

connect to AWS secrets manager.
Retrivee the username and password. You need to map it the way you have stored in secrets manager.
Store that in a variable and pass it to connection string.

Below is the sample python script provided by amazon:

import boto3
import base64
from botocore.exceptions import ClientError


def get_secret():

    secret_name = "<<{{MySecretName}}>>"
    region_name = "<<{{MyRegionName}}>>"

    # Create a Secrets Manager client
    session = boto3.session.Session()
    client = session.client(
        service_name='secretsmanager',
        region_name=region_name
    )

    # In this sample we only handle the specific exceptions for the 'GetSecretValue' API.
    # See https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_GetSecretValue.html
    # We rethrow the exception by default.

    try:
        get_secret_value_response = client.get_secret_value(
            SecretId=secret_name
        )
    except ClientError as e:
        if e.response['Error']['Code'] == 'DecryptionFailureException':
            # Secrets Manager can't decrypt the protected secret text using the provided KMS key.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'InternalServiceErrorException':
            # An error occurred on the server side.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'InvalidParameterException':
            # You provided an invalid value for a parameter.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'InvalidRequestException':
            # You provided a parameter value that is not valid for the current state of the resource.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'ResourceNotFoundException':
            # We can't find the resource that you asked for.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
    else:
        # Decrypts secret using the associated KMS CMK.
        # Depending on whether the secret is a string or binary, one of these fields will be populated.
        if 'SecretString' in get_secret_value_response:
            secret = get_secret_value_response['SecretString']
        else:
            decoded_binary_secret = base64.b64decode(get_secret_value_response['SecretBinary'])

    # Your code goes here.

The code you shared, i found it in the secrets manager, my question is how to pass the username and password into new variables and use it in the connection string. Should i execute the above code first and pass the secret keys to username and password fields ? rds = json.dumps(client.get_secret_value(SecretId="postgres")['SecretString']) — Bharat Guda, May 01 '19 at 14:16
Yes, execute above code and the below line will store that value in variable secret ..secret = get_secret_value_response['SecretString'] ... use the variable secret as a parameter to your postgres connection string. — Krunal Barot, May 01 '19 at 14:44
If you create the secret using the console with key/value pairs, it will be stored as a JSON blob. You will have to parse the JSON into a dict first and then access the key/value pairs. See Parimal's answer. — JoeB, May 01 '19 at 22:19

Connect to Postgres database using AWS Secrets Manager

2 Answers2