1

I am trying to access Timestream from EC2/Lambda instances that run within a VPC so that I can speak to a RDS instance from those EC2 instances/Lambda functions. I have spent many hours trying to get access to Timestream via PrivateLink/a VPC instance endpoint to work and think I may have found an issue. When I provision a VPC endpoint for the Timestream ingest service, the Private DNS name is specific to the cell endpoint, e.g. ingest-cell2.timestream.us-east-1.amazonaws.com NOT the general endpoint URL that boto3 uses, i.e. ingest.timestream.us-east-1.com. When I run a nslookup on ingest-cell2.timestream.us-east-1.amazonaws.com it properly resolves to the private IP of the VPC endpoint ENI, but if I lookup the more general endpoint URL of ingest.timestream.us-east-1.com it continues to resolve to public AWS IPs. The result of this is that if I initialize the timestream write client normally and perform any actions, it hangs because it is trying to communicate with a public IP from a private subnet,

import boto3
ts = boto3.client('timestream-write')
ts.meta.endpoint_url # https://ingest.timestream.us-east-1.amazonaws.com
ts.describe_endpoints() # hangs
ts.describe_database(DatabaseName='dbName') # hangs

If I explicitly give it the cell specific endpoint URL, the describe_endpoints() function throws an error but seemingly normal functions work (haven't tested writes or reads yet, just describing databses)

import boto3
ts = boto3.client('timestream-write', endpoint_url='https://ingest-cell2.timestream.us-east-1.amazonaws.com')
ts.describe_endpoints() # throws UnknwonOperationException error
ts.describe_databse(DatabaseName='dbName') # Succeeds

If I provision a NAT gateway in the private subnet rather than a VPC endpoint everything works normally as expected. Furthermore for fun, I tried adding the VPC endpoint private IP to the /etc/hosts file with ingest.timestream.us-east-1.com to force proper resolution and even then I get the same hanging behavior when running the above block of code

This seems pretty broken to me. The whole point of the VPC endpoint is to enable the SDK to operate normally. Maybe I am missing something?

AKiss
  • 11
  • 2
  • Thank you for documenting this. I am seeing the same issue with a `python3.9` lambda function running `boto3-1.20.32` `botocore-1.23.32`. I have seen issues with Timestream feature releases front running boto3 before, so I am hopeful AWS will address this in a future boto3 version. – Nadir Sidi Feb 24 '23 at 20:53
  • @NadirSidi am I right in saying that this effectively means that a single instance (EC2 or Lambda) cannot access data both from Timestream and any form of RDS database? That seems incredibly restrictive and I just can't fathom that AWS has not figured that out thusfar. – AKiss Feb 27 '23 at 13:34

0 Answers0