0

I am dealing with one problem, before starting developing, I decided to do some research.

So the problem is what would be the efficient solution when storing files?

I read about it, and some people was against storing files within database, as it will have negative impact on backups / restorations, it will add more processing time when reading database for large files and etc...

Good option would be to use S3 or any other cloud solution to store the files, but for this current customer cloud won't be good.

Another option would be to store files under file system. The concept is clear, but I try to understand what I need to understand before implementing that solution.

For example we need to consider how we structure directories, if we would store 100 000 files in one directory it can be come slow to open and etc. As well there is like maximum amount of files that can be stored in one directory.

Is there any 3rd party tools that helps to manage files in file system? that automatically partitions files and places them in directories?

LuckyLuke
  • 1,028
  • 2
  • 14
  • 28

1 Answers1

1

I work with a software that have more than 10 million files in file system, how you will structure the folders depends, but what I did was:

  1. Create a folder to each entity (Document, Image...)
  2. Into each folder create a folder to each ID object with ID beign the name of the folder, and put their files inside, but this could vary.

Example to Person that have the ID 15:

ls /storage/Person/15/Image/

Will give me this 4 images that in the database I linked to person with the ID 15:

Output:
1.jpg
2.png
3.bmp
4.bmp

If you have a HUGE amount of elements, you cold separate each digit of an ID into a subfolder, that is: Person wih ID 16549 will have this path: /storage/Person/1/6/5/4/9/Image/

About limits of files in folder I suggest you to read this thread: https://stackoverflow.com/a/466596/12914069

About a 3rd party tool I don't know, but for sure you could build this logic into your code.

English isn't my first language, tell me if you didn't understand something.

Geovanni Cassemiro Rocha
  • 86
  • 3
  • Thank you for your response. Currently, most of the stuff will be Images and PDF documents. It can quickly grow astronomically in quantity. My main concern was about the limits of the quantity of entries in directory. Probably I will go with such approach: /storage/{{YYYY-MM}}/Image/{{GUID}}.{{ext}} || /storage/{{YYYY-MM}}/Pdf/{{GUID}}.{{ext}} – LuckyLuke Mar 06 '22 at 10:32