Allow AWS Lambda to access RDS Database

Question

I am trying to connect to RDS Database from an AWS Lambda (Java).

Which IP should I enable from the RDS Security group rules?

Answer 1

You can't enable this via IP. First you will need to enable VPC access for the Lambda function, during which you will assign it a Security Group. Then, within the Security Group assigned to the RDS instance you will enable access for the Security Group assigned to the Lambda function.

Answer 2

You can configure Lambda to access your RDS instance.

You can enable this using Lambda management console. Select Lambda function which need access to RDS instance and then go to Configuration -> Advanced settings and select the VPC (which is your RDS instance is in) you need it to access.

find out more here http://docs.aws.amazon.com/lambda/latest/dg/vpc.html

Answer 3

For anyone else searching for a more detailed solution, or lambda config provisioned via AWS SAM / Cloudformation, what worked for me was:

i. create a Security Group (SG) allowing outbound traffic on the desired port you'd like to connect over (eg: 5432 or 3306. Note, inbound rules have no affect on lambda I believe, currently) Apply that SG to your lambda.

ii. create an SG allowing inbound traffic on the same port (say 5432 or 3306) which references the lambda SG, so traffic is locked down to only the lambda. And outbound on the same port (5432 or 3306). Apply that SG to your RDS instance.

Further detail:

Lambda SG:

Direction    Protocol    Port     Source
Outbound     TCP         5432     ALL

RDS SG:

Direction    Protocol    Port     Source
Inbound      TCP         5432     Lambda SG
Outbound     TCP         5432     ALL

SAM template.yaml to provision the main resources you'll probably require including: an RDS cluster (Aurora Postgres serverless to minimise running costs is shown in this example), a Postgres master user password stored in secrets manager, a lambda, an SG that is applied to the lambda allowing outbound traffic on port 5432, an SG that is applied to the RDS cluster referencing the lambda SG (locking down traffic to the lambda) and I have also shown optionally how you may wish to connect to the RDS from your local desktop machine using a desktop DB client (eg DBeaver) over SSH tunnel via a bastion (eg a nano EC2 instance with an EIP attached so it can be stopped and all config remain the same) to admin the RDS from your local machine.

(Please note for a production system you may wish to provision your RDS into a private subnet for security. Provisioning of subnets not covered here for brevity. Also please note for a production system passing a secure secret as an env variable is not best practice the lambda should really resolve the secret each time - however shown passed as an env var for brevity)

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Provisions stack with Aurora Serverless

Parameters:
  AppName:
    Description: "Application Name"
    Type: String
    Default: RDS-example-stack
  DBClusterName:
    Description: "Aurora RDS cluster name"
    Type: String
    Default: rdsexamplecluster
  DatabaseName:
    Description: "Aurora RDS database name"
    Type: String
    Default: examplerdsdbname
  DBMasterUserName:
    AllowedPattern: "[a-zA-Z0-9_]+"
    ConstraintDescription: must be between 1 to 16 alphanumeric characters.
    Description: The database admin account user name, between 1 to 16 alphanumeric characters.
    MaxLength: '16'
    MinLength: '1'
    Type: String
    Default: aurora_admin_0

Resources:
  # lambdas
  someLambda:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Sub '${AWS::StackName}-someLambda'
      # Role: !GetAtt ExecutionRole.Arn # if you require a custom execution role and permissions
      VpcConfig:
        SubnetIds: [subnet-90f79cd8, subnet-9743e6cd, subnet-8bf962ed]
        SecurityGroupIds: [!Ref lambdaOutboundSGToRDS]
      Handler: index.handler
      CodeUri: ./dist/someLambda
      Runtime: nodejs14.x
      Timeout: 5 # ensure matches your PG/ mySQL connection pool timeout
      ReservedConcurrentExecutions: 5
      MemorySize: 128
      Environment: # optional env vars useful for your DB connection
        Variables:
          pgDb: !Ref DatabaseName
          # dbUser: '{{resolve:secretsmanager:some-stackName-AuroraDBCreds:SecretString:username}}'
          # dbPw: '{{resolve:secretsmanager:some-stackName-AuroraDBCreds:SecretString:password}}'

  # SGs
  lambdaOutboundSGToRDS: # Outbound access for lambda to access Aurora Postgres DB
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: !Sub ${AWS::StackName} access to Aurora PG DB
      GroupName: !Sub ${AWS::StackName} lambda to Aurora access
      SecurityGroupEgress: 
        -
          CidrIp: '0.0.0.0/0'
          Description: lambda to Aurora access over 5432
          FromPort: 5432
          IpProtocol: TCP
          ToPort: 5432
      VpcId: vpc-f6c4ea91

  RDSSG:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: !Sub ${AWS::StackName} RDS ingress and egress
      SecurityGroupEgress: 
        -
          CidrIp: '0.0.0.0/0'
          Description: lambda RDS access over 5432
          FromPort: 5432
          IpProtocol: TCP
          ToPort: 5432
      SecurityGroupIngress: 
        -
          SourceSecurityGroupId: !Ref lambdaOutboundSGToRDS # ingress SG for lambda to access RDS
          Description: lambda to Aurora access over 5432
          FromPort: 5432
          IpProtocol: TCP
          ToPort: 5432
        - # optional
          CidrIp: '172.12.34.217/32' # private IP of your EIP/ bastion instance the EIP is assigned to. /32 ie a single IP address
          Description: EC2 bastion host providing access to Aurora RDS via SSH tunnel for DBeaver desktop access over 5432
          FromPort: 5432
          IpProtocol: TCP
          ToPort: 5432
      VpcId: vpc-f6c4ea91

  DBSubnetGroup: # just a logical grouping of subnets that you can apply as a group to your RDS
    Type: AWS::RDS::DBSubnetGroup
    Properties:
      DBSubnetGroupDescription: CloudFormation managed DB subnet group.
      SubnetIds:
        - subnet-80f79cd8
        - subnet-8743e6cd
        - subnet-9bf962ed

  AuroraDBCreds: # provisions a password for the DB master username, which we set in Parameters
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: !Sub ${AWS::StackName}-AuroraDBCreds
      Description: RDS database auto-generated user password
      GenerateSecretString:
        SecretStringTemplate: !Sub '{"username": "${DBMasterUserName}"}'
        GenerateStringKey: "password"
        PasswordLength: 30
        ExcludeCharacters: '"@/\'
      Tags:
        -
          Key: AppName
          Value: !Ref AppName

  RDSCluster:
    Type: AWS::RDS::DBCluster
    Properties:
      DBClusterIdentifier: !Ref DBClusterName
      MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref AuroraDBCreds, ':SecretString:username}}' ]]
      MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref AuroraDBCreds, ':SecretString:password}}' ]]
      DatabaseName: !Ref DatabaseName
      Engine: aurora-postgresql
      EngineMode: serverless
      EngineVersion: '10' # currently provisions '10.serverless_14' 10.14
      EnableHttpEndpoint: true
      ScalingConfiguration:
        AutoPause: true
        MaxCapacity: 2
        MinCapacity: 2
        SecondsUntilAutoPause: 300 # 5 min
      DBSubnetGroupName:
        Ref: DBSubnetGroup
      VpcSecurityGroupIds:
        - !Ref RDSSG

# optional outputs useful for importing into another stack or viewing in the terminal on deploy
Outputs:
  StackName:
    Description: Aurora Stack Name
    Value: !Ref AWS::StackName
    Export:
      Name: !Sub ${AWS::StackName}-StackName

  DatabaseName:
    Description: Aurora Database Name
    Value: !Ref DatabaseName
    Export:
      Name: !Sub ${AWS::StackName}-DatabaseName

  DatabaseClusterArn:
    Description: Aurora Cluster ARN
    Value: !Sub arn:aws:rds:${AWS::Region}:${AWS::AccountId}:cluster:${DBClusterName}
    Export:
      Name: !Sub ${AWS::StackName}-DatabaseClusterArn

  DatabaseSecretArn:
    Description: Aurora Secret ARN
    Value: !Ref AuroraDBCreds
    Export:
      Name: !Sub ${AWS::StackName}-DatabaseSecretArn

  DatabaseClusterID:
    Description: Aurora Cluster ID
    Value: !Ref RDSCluster
    Export:
      Name: !Sub ${AWS::StackName}-DatabaseClusterID

  AuroraDbURL:
    Description: Aurora Database URL
    Value: !GetAtt RDSCluster.Endpoint.Address
    Export:
      Name: !Sub ${AWS::StackName}-DatabaseURL

  DatabaseMasterUserName:
    Description: Aurora Database User
    Value: !Ref DBMasterUserName
    Export:
      Name: !Sub ${AWS::StackName}-DatabaseMasterUserName

Answer 4

Here is what I did

I assigned same Subnets and VPCs to both services Lambda and RDS. Now I created a NAT Gateway choosing one of the subnet so that Lambda can use that NAT Gateway to interact with the outside world.

Last thing is to add inbound entry in the security group that is attached to RDS as well as Lambda functions. Whitelist DB port 5432 in my case for postgresql and add security group name in the source.

Security group is somehow whitelisting itself by adding an entry in inbound rules.

This worked for me pretty well.

Answer 5

The recommended way is still (1) VPC and data-api, however you can also go with (2) which is RDS proxy (https://aws.amazon.com/blogs/compute/using-amazon-rds-proxy-with-aws-lambda/) that supports both MySQL and PostgreSQL since June 30 2020.

Answer 6

You don't need to use IP.

I assume that your RDS is in private subnet of the VPC. This means that your lambda should also be in VPC to communicate with database.

Let's assume that the credentials for your RDS is in secret manager. You can give necessary permission to the secret so that your lambda can access the decrypt secret within lambda function.

Add proper ingress rule to the database. Make sure your security groups are configured properly. You can also use RDS proxy to re-use the db connections to give better performance.

This post talks about how to communicate to RDS from lambda https://www.freecodecamp.org/news/aws-lambda-rds/