How I can get list of objects from S3 bucket via aws lambda?

Question

I have problem with AWS lambda, when I trying to get list of objects from S3 bucket. I amd doing it via boto3 paginator:

but I cannot to receive the data from paginator as results. Tricky is that I can do it on local machine, but when I try to do this via deployed aws lambda - it is just gone to the time out.

Does lambda has permissions to access S3 or are you in private subnet? — Marcin, Mar 05 '20 at 07:45
@ppostnov If private subnet, does your lambda have permissions to use private subnet? If yes, do you use VPN gateway to access S3 or have NAT setup? — Marcin, Mar 05 '20 at 07:49
This might be helpfull - https://stackoverflow.com/a/58220730/9931092 — Amit Baranes, Mar 05 '20 at 07:52
If it's timing out my guess is you need an S3 VPC endpoint, since you say the lambda's in a private subnet. Without the endpoint you will get a timeout as opposed to some other kind of error (assuming there isn't internet access via a NAT gateway or something). — 404, Mar 05 '20 at 10:28
@marcin I have direct connect vpc + 2 privat subnet + one security group — ppostnov, Mar 05 '20 at 11:24
@404 I have direct connect vpc + 2 privat subnet + one security group — ppostnov, Mar 05 '20 at 11:24
yes, it indeed related to my current vcp settings, because a lambda works properly without it. — ppostnov, Mar 05 '20 at 11:50
@ppostnov If you don't have an S3 endpoint in your VPC, add one in and see if that sorts it. — 404, Mar 05 '20 at 12:07
@404 if you mean endpoint like this com.amazonaws.eu-west-1.s3, then I already have it — ppostnov, Mar 05 '20 at 12:19
Hi @ppostnov, ensure that your private subnets have internet access ( through nat gateway is a possibility ) or as 404 mention you can use and s3 endpoint in your vpc ( https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints.html ) — pepo, Mar 05 '20 at 16:21

score 0 · Answer 1 · answered Sep 13 '21 at 08:16

I was the same problem and what worked for me is to give to Lambda policy that have the access to S3 you can see the following guide on how to do it:

https://www.anblicks.com/blog/10-step-guide-to-configure-s3-bucket-with-a-lambda-function-using-sqs

And also to increase the time out of Lambda from 3 seconds to more

How I can get list of objects from S3 bucket via aws lambda?

1 Answers1