5

I've got nearly a million images for my site and they are stored in one folder on my windows server.

Since opening this folder directly on desktop drive me and my CPU crazy I am wondering that whether fetching one of them using my PHP script for a HTTP request is also laborious.
So, will separating them into different folders improve the performance?

dotslashlu
  • 3,361
  • 4
  • 29
  • 56

3 Answers3

3

No, the performance does not depend on the number of files that are in a directory. The reason why opening the folder in Windows explorer is slow is because it has to render icons and various other GUI related things for each file.

When the web server fetches a file, it doesn't need to do that. It just (more or less) directly goes to the location of the file on the disk.

EDIT: Millions is kind of pushing the limits of your file system (I assume NTFS in your case). It appears that anything over 10,000 files in a directory starts to degrade your performance. So not only from a performance standpoint, but from an organizational standpoint as well, you may want to consider separating them into subdirectories.

tskuzzy
  • 35,812
  • 14
  • 73
  • 140
  • This is generally correct, but there are file system penalties for having a massive amount of files in one folder: see http://stackoverflow.com/questions/197162/ntfs-performance-and-large-volumes-of-files-and-directories for a decent discussion of some of the performance tradeoffs. In addition, if you are using directory listings those will be particularly slow. – Femi Aug 10 '11 at 19:16
  • @Femi: Thanks for the addendum. I edited my answer accordingly. – tskuzzy Aug 10 '11 at 19:21
1

Often the best answer in a case like this is to benchmark it. It shouldn't be too hard to create a program that opens 1000 hard-coded file names and closes them. Run the test on your million-plus directory and another directory containing only those 1000 files being tested and see if there's a difference.

Not only does the underlying file system make a difference, but accessing over a network can affect the results too.

Mark Ransom
  • 299,747
  • 42
  • 398
  • 622
0

Separating your files into seperate directories will most likely help performance. But as mark suggests it's probably worth benchmarking

Toby Allen
  • 10,997
  • 11
  • 73
  • 124
  • I've seen a case where splitting files into separate directories completely killed performance, although in this case it was splitting into a two level deep hierarchy. I wouldn't have predicted that, which is why I advocate benchmarking. – Mark Ransom Aug 10 '11 at 22:21