3

I have a Rails 3 app with Paperclip gem. Actually, my local directories structure is based on my record UUID to stock images:

5D5E5641-FCE8-4D0B-A413-A9F993CD0E34

becomes:

5/D/5/E/5/6/....... 3/4/full/image.jpg
5/D/5/E/5/6/....... 3/4/thumb/image.jpg

so that, I never have more than 32000 nodes per directory.

I want to migrate to S3: 1) Can I keep this directories structure on S3 ? Could it be a perf issue ? 2) Does Amazon S3 has its own directories management per bucket ?

Thanks.

alex.bour
  • 2,842
  • 9
  • 40
  • 66

1 Answers1

0

There is no such stuff as folders in Amazon S3. It is a "flat" file system. The closer you can get to folders is adding prefixes like you said: 5/D/image.jpg to your file names. In this case, 5 is a prefix and 5/D is also a prefix. On the other hand, your delimiter could be /.

Even though several S3 tools will show you stuff as if they were contained inside folders, this concept does not exist on S3. Please see this and this related threads.

You can definitely use the pattern you suggested, and I don't think you will suffer any performance penalties by doing so.

Community
  • 1
  • 1
Viccari
  • 9,029
  • 4
  • 43
  • 77
  • Someone told me that using a hierarchical structure would even *increase* performance, since under the hood, there is a unix-based filesystem. So, when you use slashes in the file name, S3 actually stores it in a directory with that path. That way, you avoid having a directory with too many files. Is that true? – Otavio Macedo Nov 12 '12 at 16:03
  • 2
    @OtavioMacedo I don't think so. There is no reference to this kind of information in their documentation. Also, please have a look at http://stackoverflow.com/questions/394198/max-files-per-directory-in-s3 – Viccari Nov 12 '12 at 17:29