0

I want to download a set of csv files from a S3 subfolder with example path s3://folder/subfolder/date=2023-03-20. The subfolder has folders containing these csv files, which are created everyday with pattern: /date=<todays date>/. I am unable to set this pattern in the prefix argument in s3sync function. Is there a way to add this pattern?

This is my code in R:

key <- "ABCD" # access key ID
secret <- "XYZ" #secret key

Sys.setenv("AWS_ACCESS_KEY_ID" = key,
       "AWS_SECRET_ACCESS_KEY" = secret,
       "AWS_DEFAULT_REGION" ="Global")

### get todays date
today <- as.character(paste('date=',Sys.Date(),sep=""))
today

s3sync (path=".", bucket="s3://my_bucket",prefix="folder/subfolder/today ", region="Global", direction="download")
Phil
  • 7,287
  • 3
  • 36
  • 66
Sandhya
  • 1

1 Answers1

0

I think you are almost there. Try:

myprefix <- paste0("folder/subfolder/", today)
s3sync (path=".", bucket="s3://my_bucket",prefix=myprefix, region="Global", direction="download")
Leon Samson
  • 405
  • 3
  • 12
  • Hi Leon. Thanks for the reply. I tried doing this, but it says '0 files to sync'. It works when I put prefix as prefix="Folder/subfolder/'. It lists out all the folders present in the subfolder, which I do not want. I only want the subsubfolder with current date. As mentioned in my description above, the subfolder contains another set of subsubfolders in the filename format: "data=, data=, and so on. – Sandhya Mar 20 '23 at 12:16