Connect mongodb from AWS lambda python using IAM

Question

EDIT 3 FOR BOUNTY :

The below error is not relevant anymore, pymongo accepts the encoded connection string but this is not the good way to go with the AWS Lambda according to this documentation.

I am trying to connect my AWS Lambda to my Atlas cluster database using pymongo[aws] but it times out all the time.

The URI should look like this according to the above documentation : uri = "mongodb://example.com/?authMechanism=MONGODB-AWS"

I have tried it of course, and i have no more errors or anything, the lambda simply times out.

I have double checked the ARN role for my lambda and the one setup in MongoDB when i have created my database user.

I have also granted the dbAdmin role to be sure it's not a permission issue but still times out.

The mongoDB community support also tries to help on this case.

Any ideas where it can come from ?

Old question

I was using until now pymongo[srv] to connect my AWS lambda to my MongoDB cluster with :

mongodb+srv://username:password@cluster.name/database

Now i am trying to setup the IAM role connection and in this repo there is a connection string such as :

"mongodb://<access_key>:<secret_key>@mongodb.example.com/?authMechanism=MONGODB-AWS&authMechanismProperties=AWS_SESSION_TOKEN:<security_token>"

I have tried this one, but i got the following error in my Cloudwatch logs :

[ERROR] ValueError: Port must be an integer between 0 and 65535: 'Fizjqbxairn6K19Fsalbucyz'
Traceback (most recent call last):
  File "/var/lang/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/var/task/lambda_function.py", line 13, in <module>
    client = MongoClient(
  File "/var/task/pymongo/mongo_client.py", line 736, in __init__
    res = uri_parser.parse_uri(
  File "/var/task/pymongo/uri_parser.py", line 568, in parse_uri
    nodes = split_hosts(hosts, default_port=default_port)
  File "/var/task/pymongo/uri_parser.py", line 376, in split_hosts
    nodes.append(parse_host(entity, port))
  File "/var/task/pymongo/uri_parser.py", line 137, in parse_host
    raise ValueError("Port must be an integer between 0 and 65535: %r" % (port,))

So i am guessing that the String syntax is not correct.

Please tell me what is the correct connection string to be used in using the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY env variables :

client = MongoClient(conn_string)

Thanks in advance

EDIT :

My python string is :

client = MongoClient('mongodb://' + os.environ.get("AWS_ACCESS_KEY_ID") + ':' + os.environ.get("AWS_SECRET_ACCESS_KEY") + @clustername/databasename?retryWrites=true&authMechanism=MONGODB-AWS)

And i am installing pymongo[aws] as dependency instead of pymongo[srv].

EDIT 2 :

I confirm that the AWS_SECRET_ACCESS_KEY contains / as characters.

Answer 1

I think you should escape values in userName/password/sessionToken (AWS_*), search for aws here. In other words, all values you provide in connection string and that may contain special characters like : or / should be escaped. See this SO question about how it can be done in python

Answer 2

There are 3 moving parts that should be configured to work together:

1. python code/libraries
2. AWS Lambda role and VPC
3. Atlas Network and database access

python

starts from pip install "pymongo[aws]"

at the time of answering it pulled following dependencies:

boto3==1.26.121
botocore==1.29.121
dnspython==2.3.0
jmespath==1.0.1
pymongo==4.3.3
pymongo-auth-aws==1.1.0
python-dateutil==2.8.2
s3transfer==0.6.0
six==1.16.0
urllib3==1.26.15

The minimal lambda code to test:

from pymongo.mongo_client import MongoClient
from pymongo.server_api import ServerApi


def lambda_handler(event, context):

    uri = "mongodb+srv://<cluster_name>.<subdomain>.mongodb.net/?authSource=%24external&authMechanism=MONGODB-AWS&retryWrites=true&w=majority"
    # Create a new client and connect to the server
    client = MongoClient(uri, server_api=ServerApi('1'))
    # Send a ping to confirm a successful connection
    try:
        client.admin.command('ping')
        print("Pinged your deployment. You successfully connected to MongoDB!")
    except Exception as e:
        print(e)

The cluster name and domain name in the uri can be found on Atlas connection settings. On the screenshot it's the python driver, but the domain is the same regardless of client:

Note: pymongo-auth-aws plays well with AWS environment variables, so no need to set them explicitly in the uri

AWS Lambda and Mongodb Atlas

Network access

The function should be deployed within VPC to let VPC peering with Atlas, or at least have a fixed egres IP to allow network connections to Atlas from this IP (circled blue on the image below). For the sake of experiment - temporarily allow access to Atlas from everywhere 0.0.0.0/0 to debug authentication in isolation. Once you get it connected on app layer, return back to the network configuration. Don't leave it open even on dev environment

Data base access:

The authentication is based on the role name, so you need to copy exact ARN of the lambda execution role, create a user on Atlas side, paste the role name, and assign mongodb level permissions to the user (red path on the image below).

Testing:

Once it's configured, you can trigger the function manually (payload doesn't matter, it's not being used), and observe Pinged your deployment. You successfully connected to MongoDB! in the log: