3

I have a use case where I need to download bucket files within a particular modified date range only for diagnosis purpose. There is a large amount of files in the bucket so it is not pracitcal to download all files for the purpose. I have looked at the AWS CLI and it seems there is no readily available S3 options for this (there is only a "--include" option which filters by file name only). May I seek for your advice on this? Thank you

Patrick C.
  • 1,339
  • 3
  • 16
  • 31
  • 1
    run an S3 inventory or a couple of list requests to get information on all the objects, do the date filtering on your client and then download the relevant files. – luk2302 May 24 '21 at 06:16
  • Use s3api . Refer https://stackoverflow.com/questions/45429556/how-list-amazon-s3-bucket-contents-by-modified-date – Jyothish May 24 '21 at 06:28
  • There is no such capability. You would need to write your own code to pick the objects you wish to download. However, if you are wanting to download the _most recent_ objects, there is a shortcut for getting this list by using a `--query` parameter. How will you be selecting the date range? – John Rotenstein May 24 '21 at 07:31

1 Answers1

-1
DATE=$(date +%Y-%m-%d)

FILE="$(aws s3api list-objects --bucket bucket_name --query 'Contents[?LastModified>=  `'"$DATE"'`][].{Key: Key}' --output=text| grep file_name)"

aws s3 cp s3://bucket_name/${FILE} .
Mohitd23
  • 1,439
  • 2
  • 12
  • 10