5

I am working on a web application and i need to store images on server (these uploaded by user). i am generating new Guid for every new image.And i am creating three images of one image i.e. Original,icon and Thumb. still i am using three different folders to save these.My Project manager ask me why i haven't create different folders for every users. So my question is which is better way to store images between the following:

1). Upload images in only three folders i.e. Original, Icon and Thumb, or 2). create folder for every user and then create three folders in it for original,thumb and icon. one thing i would like to ask if the number of users will more than 100000 then will it effects performance???

which is better approach. please help me to take decision. Thanks

Ram
  • 1,131
  • 10
  • 28
  • 52
  • How are you retrieving the images? do you have their path stored in database ? – Habib Oct 17 '12 at 10:36
  • no still i am not storing path in db. if i will need to change the process means shift to create different folders for every user then i will do this. – Ram Oct 17 '12 at 10:38

3 Answers3

10

I'd store images by user. However, given that you potentially have a lot of users, it may not be that simple.

User file uploads are just that: user-specific file uploads. When you manage these files you often need to apply various procedures per-user. For example: 1) remove user account and delete all related files, 2) calculate amount of space a user uses, 3) list all files a user has uploaded, etc.

If you scatter these files across various directories it is much harder to efficiently implement the above procedures.

You specified that you may have more than 100.000 users. Depending on what your filesystem is, you may end up having troubles because of this. For instance in ext3, there is maximum of 32K subdirectories per directory, which can be a problem for organizing the files in directories per user (see: How many files can I put in a directory?).

Assuming that we cannot store more than 32K files or directories inside a directory then you would need to find out a way to work around this limit. For instance, you can split user folders into several subdirectories according to the first letter in username (and add an extra subdir for all the other starting characters):

users
   a
      aaron
         ...
   b       
      bertie
         ...
   ...
   misc
      _foobar 
         ...
      @jack
         ...

Now you only have roughly 100000/25=4000 users per directory, which is under the given 32K.

Thinking outside of the box, it may not be the right choice to store the images as files in a flat filesystem. There are database systems that are specifically created for storing and handling large numbers of files. Take MongoDB's GridFS for instance. It is very efficient and scales to large number of files, also taking care of all the lower-level issues like correct filesystem usage. In MS SQL there is the FILESTREAM Storage, which is specifically good at storing files on an NTFS filesystem.

Community
  • 1
  • 1
jsalonen
  • 29,593
  • 15
  • 91
  • 109
3

It depends. For security reasons I would prefer to store images by user folder. If security is not relevant, I would store in three different folders, because of the number of folders.

Update: Every creation of a folder forces ASP.NET to recompile the application pool, with every consequences, e.g. session loosing.

I recommend to create folder structure like that: 

year
  month
    day
      hour - if necessary
        images

This means only one folder per hour has to be created. But with this solution it is necessary to save image path for every user.

Kai
  • 1,953
  • 2
  • 13
  • 18
  • 1
    Why? Minimizing the number of directories alone is not IMHO a good argument. If you have >32000 images, you end up in trouble if they are all stored in the same directory (see: http://stackoverflow.com/questions/466521/how-many-files-in-a-directory-is-too-many). Then you end up fixing the problem by adding subfiles (like images/a/, images/b/, etc.), what I would call a mess. Been there. – jsalonen Oct 17 '12 at 10:42
  • @jsalonen: You're absolutely right. You will get a problem with so many files. But every creation of a new folder forces ASP.NET to recompile the application pool (depends on settings), with every consequences, e.g. session loosing. – Kai Oct 17 '12 at 10:47
  • Thanks for the crucial extra info. Note that the asker also told that there may be more than 100K users, which makes this a more complicated issue. – jsalonen Oct 17 '12 at 10:49
  • 1
    edited my answer...In my experience, this solution is sufficiently – Kai Oct 17 '12 at 10:56
0

I would store it by user, but I would not create extra folders for each image size. I would just store the three images directly in the user folder.

Also, I would store all folders outside the project folder as one way to avoid application restart when new folders are created.

Mario S
  • 11,715
  • 24
  • 39
  • 47
  • one thing i would like to ask if the number of users will more than 100000 then will it effects performance??? – Ram Oct 17 '12 at 10:44
  • @Visions I do not believe you will notice any performance difference for 100000 folders, unless you iterate all of them at once. – Mario S Oct 17 '12 at 10:52