In my S3 bucket, I have a folder structure of rootFolder1/year/month/day/myfile.csv
. Depending on the time, the rootFolder
folder could have multiple subfolder, organised by year
, month
and date
, with each date
folder containing multiple csv files. There could also be a rootFolder2
in my bucket, with the same folder structure.
I know for downloading object from S3, I can use the getObject
method where I can pass in the params
object with the bucket Key
like so:
var params = {
Key: "book/2020/01/01/all_books_000.csv",
Bucket: 'my-bucket-test'
}
s3.getObject(params, (err, data) => {
if (err) throw err;
// do something with data
})
But in my case I don't want to hardcode the Key
in. Is there any elegant way I can call getObject
where I can dynamically set the Key
, so it downloads the files under each date
folder?
edit: for some further clarity, these are the keys I get back when I call s3.listObjects
'author/',
'author/2020/',
'author/2020/01/',
'author/2020/01/01/',
'author/2020/01/01/all_authors_000.csv',
'book/',
'book/2020/',
'book/2020/01/',
'book/2020/01/01/',
'book/2020/01/01/all_books_000.csv',
'book/2020/01/01/all_books_001.csv'