0

Hi I need a way to get size of individual key in a bucket with specific prefix. I want to search for a file that is smaller than some threshold. What would be the most efficient way to do it? I will need to execute this numerous times per day.

  • You can use a HEAD request (see http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectHEAD.html ) to retrieve metadata for a file – Marian Theisen Aug 30 '17 at 11:33

2 Answers2

0

This link has good info - Check file size on S3 without downloading?

To do this in an automated fashion you can either write it as a Bash script and schedule it via Crontab on your own server or use the AWS SDK in your preferred language on Lambda and then schedule it via CloudWatch Events using a Cron task.

http://docs.aws.amazon.com/lambda/latest/dg/with-scheduled-events.html

BryceH
  • 2,740
  • 2
  • 21
  • 24
0

if you mount the bucket prefix with something like goofys, you can use find(1) to find the files you want:

$ find /mnt/s3bucket -size -1048576c
khc
  • 344
  • 2
  • 8