1

I have to replicate my local folder structure in S3 bucket, I am able to do so but its not creating folders which are empty. My local folder structure is as follows and command used is.

"aws-exec s3 sync ./inbound s3://msit.xxwmm.supplychain.relex.eeeeeeeeee/enter image description here

its only creating inbound/procurement/pending/test.txt, masterdata and transaction is not cretated but if i put some file in each directory it will create.

Ajeet
  • 68
  • 1
  • 9
  • Why do you wish to replicate the folder structure? Folders will automatically 'appear' when you copy files to them, even if they don't already exist. – John Rotenstein May 18 '17 at 17:14
  • we have a requirement like that, we follow a standard directory structure, we want few files to be uploaded on S3 and run the EMR job to calculate difference, but at the same time i want same directory structure which is on my local server. – Ajeet May 19 '17 at 17:41

2 Answers2

2

As answered by @SabeenMalik in this StackOverflow thread:

S3 doesn't have the concept of directories, the whole folder/file.jpg is the file name. If using a GUI tool or something you delete the file.jpg from inside the folder, you will most probably see that the folder is gone too. The visual representation in terms of directories is for user convenience.

Community
  • 1
  • 1
Yeshodhan Kulkarni
  • 2,844
  • 1
  • 16
  • 19
2

You do not need to pre-create the directory structure. Just pretend that the structure is there and everything will be okay.

Amazon S3 will automatically create the structure as objects are written to paths. For example, creating an object called s3://bucketname/inbound/procurement/foo` will automatically create the directories.

(This isn't strictly true because Amazon S3 doesn't use directories, but it will appear that the directories are there.)

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
  • Thanks John, it helped I dont have enough reputation to vote up to your answer, am guilty. – Ajeet May 24 '17 at 10:16