Lambda Timeout while communicating with S3

Question

I'm trying to simply list all the files in an S3 bucket using Lambda

The code looks as follows:

var AWS = require('aws-sdk');
var s3 = new AWS.S3();

exports.handler = (event, context, callback) => {

   s3.listObjectsV2({
       Bucket: "bucketname",
   }, function(err, data) {
       console.log("DONE : " + err + " : " + data); 

       callback(null, 'Hello from Lambda');
    });
};

Using the above, I never get the "DONE" printed at all. The log doesn't show any information except for the fact that it timed out.

Is there any troubleshooting I could do here? I would've thought that at least the error would've been shown in the "DONE" section.

Is your lambda function running inside a VPC? If so, does its subnet have access to the Internet via a NAT instance or NAT gateway, or direct S3 access via an S4 VPC endpoint? — Michael - sqlbot, Aug 16 '16 at 00:33
Possible duplicate of [Adding AWS Lambda with VPC configuration causes timeout when accessing S3](http://stackoverflow.com/questions/35423246/adding-aws-lambda-with-vpc-configuration-causes-timeout-when-accessing-s3) — Mark B, Aug 16 '16 at 02:10

score 22 · Answer 1 · answered Aug 16 '16 at 02:05

22

Thanks to Michael above. The problem was that it was running inside a VPC. If I change it to No VPC, it works correctly. Your solution may be different if you require it to run in a VPC.

answered Aug 16 '16 at 02:05

Zeus

569
1
7
13

score 15 · Answer 2 · answered Sep 23 '19 at 15:17

15

If you are running your code inside VPC make sure to create VPC Endpoint.
Here is the tutorial: https://aws.amazon.com/blogs/aws/new-vpc-endpoint-for-amazon-s3/

answered Sep 23 '19 at 15:17

Dean Koštomaj

598
5
12

You saved my time thank you. Have you done any course on AWS? the whole thing is too complex for me to understand. – princebillyGK Jun 02 '21 at 11:03
No, I have not done any courses on AWS. It was just a lot of research. :) – Dean Koštomaj Jun 02 '21 at 11:39
this is correct, but to save someone the frustration, do make sure you create the endpoint/route_table association as well https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint_route_table_association – sbeam Aug 20 '21 at 16:17
I believe this is only needed if the subnet is private ie has no internet gateway – Migwell Oct 02 '21 at 06:09

score 8 · Answer 3 · answered Mar 23 '18 at 10:33

8

If you are running your code inside the VPC, Make sure VPC subnet and its routing table entry should be proper (routing : Dest= 0.0.0.0/0 and target = igw-xxxx). Also VPC endpoint routing must be added in order to communicate to s3 via endpoint.

answered Mar 23 '18 at 10:33

palani.p

521
1
7
13

2

This will help you create VPC endpoint: https://aws.amazon.com/blogs/aws/new-vpc-endpoint-for-amazon-s3/ – Dean Koštomaj Sep 23 '19 at 15:18
This final part is what I was missing "Also VPC end point routing must be added in order to communicate...". When you create a vpc endpoint it doesn't by default choose a routing table. Chances are if you're using the 'default' vpc for your aws account, it already has a routing table. Click into the vpc endpoint you just created & choose 'manage route tables' then just choose the route table associated with the vpc. Done! FYI, didn't try 'interface' option when setting up the VPC endpoint in the first place, but 'gateway' worked for me. – Reece Jul 27 '22 at 16:54

score 8 · Answer 4 · answered Dec 14 '18 at 11:38

8

In my case I have selected 2 different subnets, 1 is private and other is public. So it was working sometimes and sometimes not. I changed both subnets to private (having NAT gateway in route) and now it that worked without timeout error.

answered Dec 14 '18 at 11:38

ExploringApple

1,348
2
17
30

This was exactly my case: I added lambda to all 4 subnets (2x public, 2x priv). Only priv had the networking configured properly. Turning off public assignment solved problem of conn/timeout – 108adams Jul 24 '22 at 10:41

score 0 · Answer 5 · answered Jun 15 '23 at 18:13

I had the same issue.

My lambda function was inside a private subnet within a VPC.

I spawned a NAT instance in the public subnet of the same VPC.

Defined the necessary rules for the security groups of the lambda function and NAT instance.

Thus using a NAT instance solved my issue, and it was a cheaper solution for me compared to a NAT Gateway.

Lambda Timeout while communicating with S3

5 Answers5

Linked