8

I'm trying to set up an Aurora Postgres target endpoint on AWS DMS and I'm getting the error below when testing. There are no special permissions set up on the secret. The IAM role has SecretsManagerReadWrite attached and trusts dms.us-west-2.amazonaws.com. Attached below is a screenshot of the setup screen.

Note that I do currently have one (regular Postgres) target endpoint already set up, and I get the same error when trying to just recreate that one. Any advice would be appreciated, I'm tearing my hair out! I'm happy to provide more info, just not sure what else is needed.

Error message: Test Endpoint failed: Application-Status: 1020912, Application-Message: Failed to retrieve secret. Unable to find Secrets Manager secret, Application-Detailed-Message: Unable to find AWS Secrets Manager secret Arn 'arn:aws:secretsmanager:us-west-2:###########:secret:data_modeling_db/pipelines_write_user-#####' The secrets_manager get secret value failed: curlCode: 28, Timeout was reached Too many retries: curlCode: 28, Timeout was reached

Endpoint creation screenshot: endpoint creation

Chris Stephens
  • 201
  • 2
  • 7
  • 1
    Is your DMS instance in a private subnet without a NAT gateway or a VPC without an internet gateway? – jordanm Jun 28 '21 at 04:25
  • good question - not quite sure how to check the private subnet part, but it's in the same VPC we're using for our prior endpoint that does work, so I don't think that's the problem. – Chris Stephens Jun 28 '21 at 18:01

2 Answers2

11

SOLVED (thanks to AWS Support)

  1. Create VPC endpoint: VPC > Endpoints > Create Endpoint > select Service com.amazonaws.us-west-2.secretsmanager > select VPC > check Enable DNS name > select Security Group(s) > Create Endpoint
  2. Get DNS name for endpoint: VPC > Endpoints > select Endpoint > copy DNS name for region/subregion (I just used the top one)
  3. Create DMS Endpoint: DMS > Endpoints > Create Endpoint > set up as normal > Endpoint Settings > check Use endpoint connection attributes > paste "secretsManagerEndpointOverride="
  4. Tada!
Chris Stephens
  • 201
  • 2
  • 7
  • 2
    For future readers, this should only be needed if the subnet does not have internet access (no internet gateway and/or NAT gateway in the route table of the subnet). – jordanm Jun 29 '21 at 16:08
  • 1
    There's now a big note. You have 1 instance that uses AWS DMS version 3.4.7. Upgrades to AWS DMS versions 3.4.7 and higher require that you configure AWS DMS to use VPC endpoints or use public routes. This requirement applies to source and target endpoints for these data stores: S3, Kinesis, Secrets Manager, DynamoDB, Amazon Redshift, and OpenSearch Service. Learn more -> https://docs.aws.amazon.com/dms/latest/userguide/CHAP_VPC_Endpoints.html – f01 Oct 14 '22 at 16:15
  • https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Security.html Set the endpoint extra connection attribute, secretsManagerEndpointOverride=secretsManager endpoint DNS to provide the secret manager VPC endpoint DNS, as shown in the following example. "secretsManagerEndpointOverride=vpce-1234a5678b9012c-12345678.secretsmanager.eu-west-1.vpce.amazonaws.com" – Sats Dec 02 '22 at 10:23
  • When I try to use that variable I get "SYSTEM ERROR MESSAGE:Unsupported value 'secretsManagerEndpointOverride' for extra connection attributes" – ronald mcdolittle Feb 17 '23 at 19:15
0

In my case, I got a hint from https://stackoverflow.com/a/68421722/422842. The field Secret ID is wrongly documented. It needs the secret name, not the secret ARN.

f01
  • 1,738
  • 1
  • 18
  • 21
  • 1
    I think the issue is that the call failed because the dms endpoint could not reach the secretsmanager, the call timed-out. For the configuration above, the arn is required. – Ross Bush Dec 16 '22 at 16:30
  • I don't think it's true anymore, looks like DMS is trying to catch that issue and if the ARN is provided, then it's fine, but if you just provide secret name, it will try to find a secret on the current region and build ARN by itself. I'm basing that on error messages from DMS, which provide the correct ARN in both cases – PatrykMilewski Jan 13 '23 at 09:38