2

I am currently writing a caching system that will hold serialized (json) data on disk and in memory in order to reduce I/O load on a database.

The system will work by holding the last X number of accessed files in memory and read other files from disk.

I have read that there are systems out there that reduce I/O load on nfs (which I may use in the future) systems by opening files by inode.

My questions are:

  1. Is there a way to open files on a nfs file system by inode in nodejs? If not, what homework would I need to do to make it happen?
    2. Is it absolutely impossible to open a file on a local file system by inode?
    3. if it is in fact impossible is there a faster way to reopen a file as it seems unnecessarily repetitive to have the OS stat the file over and over?
work4liberty
  • 158
  • 10
  • 2
    May I suggest you look into a tried and tested approach like memcached or redis before you embark on this adventure? Both can be used as a LRU cache, which should be perfect for you since I gather you will rebuild the data from your database? – Linus Thiel Apr 24 '12 at 16:08
  • Thank You Linus! memcached is almost exactly what i had planed. I still think i should fall back to a nfs store if a key is not in memcached. so part of my question still remains. – work4liberty Apr 24 '12 at 23:17

1 Answers1

3

  1. No, there is no user-accessible way to open files by inode, because doing so would, in some cases, allow users to bypass filesystem ACLs.

  2. Yes. Same reason.

  3. Most competent NFS clients, including the Linux kernel, will cache stat results locally.

  • 1
    not trying to troll here just want to show that my question was not totally nutz. The big boy's apparently do this trick with some kernel hackery. ex: http://perspectives.mvdirona.com/2008/06/30/FacebookNeedleInAHaystackEfficientStorageOfBillionsOfPhotos.aspx "They have extended the Linux kernel to allow NFS file opens via inode number rather than filename". When i first made my post i did not know that a kernel hack was necessary to make this happen. So unless anyone knows of a hacked nfs system that i cold use i will just live without it FTM. – work4liberty Apr 25 '12 at 19:14