8

I am working with Magento, and there is a function that merges CSS and Javascript into one big file.

Regardless the pros and cons of that, there is the following problem:

The final file gets cached at multiple levels that include but are not limited to:

  • Amazon CloudFront
  • Proxy servers
  • Clients browser cache

Magento uses an MD5 sum of the concatenated css filenames to generate a new filename for the merged css file. So that every page that has a distinct set of css files gets a proper merged css file.

To work around the caching issue, I also included the file modification timestamps into that hash, so that a new hash is generated, everytime a css file is modified.

So the full advantages of non revalidative caching score, but if something gets changed, its visible instantly, because the resource link has changed.

So far so good:

Only problem is, that the filenames that are used to generate the has, are only the ones that would normally be directly referenced in the HTML-Head block, and don't include css imports inside those files.

So changes in files that are imported inside css files don't result in a new hash.

No I really don't want to recursively parse all out imports and scan them or something like that.

I rather thought about a directory based solution. Is there anything to efficiently monitor the "last change inside a directory" on a file system basis?

We are using ext4.

Or maybe is there another way, maybe with the find command, that does all the job based on inode indexes?

Something like that?

I have seen a lot of programs that instantly "see" changes without scanning whole filesystems. I believe there are also sort of "file manipulation watch" daemons available under linux.

The problem is that the css directory is pretty huge.

Can anyone point me in the right direction?

kingjeffrey
  • 14,894
  • 6
  • 42
  • 47
The Surrican
  • 29,118
  • 24
  • 122
  • 168

4 Answers4

7

I suggest you use php-independent daemon to modify change date of your main css file when one of dependent php files are modified. You can use dnotify for it, something like:

dnotify -a -r -b -s /path/to/imported/css/files/ -e touch /path/to/main/css/file;

It will execute 'touch' on main css file each time one of the files in other folder are modified (-a -r -b -s = any access/recursive directory lookup/run in background/no output). Or you can do any other action and test for it from PHP.

XzKto
  • 2,472
  • 18
  • 18
  • best solution so far! i really like it. i was hoping for a more independet solution that does not require a background job to be started to listen, but i really like it. – The Surrican Jul 22 '11 at 14:40
  • 1
    Well, PHP is usually much slower then compiled programms and doing checks on each page load is much more expensive then triggering an acion once per change(because changes are rare). But if you are not concerned about speed - you can use php's readdir() function to loop through your folder and use php's stat() function on each file to get mtime(last modification time). You can greatly increase performance if there is some delay allowed before changes should be applied - you can run this check only each N minutes (you can use database/temporary file/etc.. to save last check time) or use cron. – XzKto Jul 22 '11 at 15:09
2

If you use the command 

ls -ltr `find . -type f `

It will give you a long listing of all files with the newest at the bottom.

Scott C Wilson
  • 19,102
  • 10
  • 61
  • 83
  • pretty good idea. you can also only read the last line with tail and directly pipe it to md5sum... however it reads all the files with find and passes them to ls, which pretty soon ends in an "Argument list too long" error for /bin/ls. of course you can raise the limit, but piping around such huge data is probably problematic... there is also an recursive switch for ls, problem there is that the sorting only works per directory and not overall :/ – The Surrican Jul 22 '11 at 11:10
2

Try to have a look to inotify packages that will allows you to be notified eah time a modification occurs in a directory.

ChrisMoll
  • 365
  • 1
  • 12
Cédric Julien
  • 78,516
  • 15
  • 127
  • 132
  • i am aware of that inotify support but i was hoping for something that doesn't need to run constantly in background but can be processed efficiently whith every page load. – The Surrican Jul 22 '11 at 14:42
  • @Joe Hopfgartner : I use inotify in python and C with one thread listening to the kernel notifications, and this is very efficient ;) but i'm not doing a webserver... – Cédric Julien Jul 22 '11 at 14:45
1

I've never used it, but apparently there is inotify support for PHP.

(inotify would be the most efficient way to get notifications under Linux)

Damon
  • 67,688
  • 20
  • 135
  • 185
  • 1
    i am aware of that inotify support but i was hoping for something that doesn't need to run constantly in background but can be processed efficiently whith every page load. – The Surrican Jul 22 '11 at 14:41