21

So Caching of course is what confuses me the most in Magento, as it does for most others I am sure. Currently one of the sites we work on is on Enterprise and utilizes FPC of course. The problem is, we have an inventory update that runs every 15 minutes. A lot of orders are placed to CSR's over the phone and through a catalog into an external system outside of Magento.

Every 15 minutes a script is ran to check any inventory in that system and to see if it differs then what is in Magento. If there is a difference then the inventory is updated in Magento. Using all Magento methods, no sql or anything like that.

We have always had caching issues and have tried all of the latest techniques when they come out. The latest one we are trying is Redis, and we have had good success on other sites with it. However, we are still seeing crazy load on the server and it is apparent that pages aren't cached.

After digging into the code it appears that after ever every model save or admin product controller save it looks to see if cache needs to be invalidated. It appears that changing any attribute, well at least inventory will mark FPC as needing to be invalidated.

I am confused about what invalidation means, because a while back we had a question out to customer support about something similar and this was the response

Full Page cache will get to invalidated state upon any changes on the products, categories, CMS even when the stock is decreased after a sale.

Now when full page cache gets to invalidated state this does not mean that something is changed on your frontend however any changes applied after the last refresh will not be shown on the frontend.

However if having the FPC validated at all times is a must for your business logic you could certainly set your Magento Installation to refresh it automatically through cron functionality as often as you desire.

However on all of the tests that I have done, on both 1.9 and 1.11 Enterprise, it appears when FPC is invalidated, the response isn't being pulled in from cache. Which is contradictory to what they have said about it just not having the newer updates.

Is there something I am missing? Does anyone have a good explanation for how the invalidation works in Magento specifically for FPC or any good links to fully understand the process and the code?

You can try this yourself for any page that is full page cached. But it is my understanding that the method processRequest in /app/code/core/Mage/Core/Model/Cache.php should set the body content with the cached response and return true if the page is cached.

To test go to any page make sure you got it cached and returning true. Go in and edit a product, in our case quantity. This will invalidate FPC. However now when you load the page that was cached before it will return false in this method and not be a cached page. I don't know if this is accurate to be able to tell if a page is cached or not but that is where my investigation lead me. Please correct me if I am wrong.

UPDATE: Upon further investigation I have found that when you save a product in the admin, the controller action

Mage_Adminhtml_Catalog_ProductController::saveAction()

will call the following method

Mage::getModel('catalogrule/rule')->applyAllRulesToProduct($productId)

Then in the Mage_CatalogRule_Model_Resource_Rule class, the applyAllRulesForDateRange method is called and that fires off the event

catalogrule_after_apply

Which the Full Page Cache module is observing and firing the clean cache method for the FPC tag. Essentially deleting all FPC cache records.

I don't see why this is necessary if previous to this the logic is clearing the FPC records that are tied to the product and category tags. Is this a bug?

dan.codes
  • 3,523
  • 9
  • 45
  • 59
  • I know this doesn't directly answer your question, but I just want to point out that your performance shouldn't be any worse than if all orders were placed through Magento, since after every sale your stock would decrease thereby invalidating your cache the same as your cron job does. – nachito Jun 12 '12 at 02:37
  • True I see your point there. Then its just not making any sense as to why the page isn't actually cached when it first gets invalidated. – dan.codes Jun 12 '12 at 02:47
  • This isn't the answer you want, but I have spoken with other Magento developers, and not one of us has had real success with the stock FPC. It just causes us problems. Typically, we end up using Varnish or some other reverse proxy cache. Keeping it separate from the internals, which are complicated enough already. – Tim Reynolds Jun 12 '12 at 14:54
  • Yeah, we have done something similar on another client site by caching with nginx. We are investigating Varnish as well but were pretty confident the Redis implementation was going to be pretty useful – dan.codes Jun 12 '12 at 15:22
  • +1 for Varnish, we use it to great success on a number of sites. The Phoenix Varnishcache extension makes using Varnish much easier by taking care of cache headers, invalidation on product save, etc. – Jim OHalloran Jun 13 '12 at 22:12
  • Yes, Varnish is the great service, it will save you a lot of pain. Varnish allows you to define your own caching rules, lifetimes. It has grace mode, once backend is not responding at least a part of your site is still there. But the nices feature is if 1000 same requests come to site at same moment, then varnish does just one request to backend server, and the rest of them are waiting, once backend returns the response varnish sends response to all 1000. Disadvantage is you need to know how to set it up. – Jaro Jun 15 '12 at 09:36
  • I got an answer for my issue from your question. +1. Thanks..! – Elavarasan Jan 09 '15 at 12:05
  • @dan.codes How do you fix the issue? after digging into the code I just arrived to your same conclusion ... even I change the observer, linked to catalog rules, to clean only that precise product cache the fpc is still wiped ( when I save a product in backend ) – WonderLand Apr 20 '18 at 09:12

4 Answers4

2

FPC is observing inventory changes because the intent is to display out of stock for any products that have been decremented to zero stock. The fix would be to create an event dispatch when a product hits zero rather than every time any product changes stock and rewrite FPC to observe that event instead of the original.

Another method would be to only invalidate the portions of cache pertaining to the products being updated, but this would be a rather significant architectural change.

gazarsgo
  • 104
  • 2
  • 12
2

You shall create a new custom indexer Mage_Index_Model_Indexer_Abstract , and create a new resource model api methods with cron jobs

Adrian Romero
  • 537
  • 6
  • 13
0

the phoenix Page Cache module clears the product pages and category pages but leaves some areas of cache invalidation uncovered. also it doesn't deal well with dynamic content.

maybe you should check out the aoe_static module which does a great job loading dynamic content by loading a default layout and rendering the blocks with an ajax call. this ajax call also sets the cookie to allowing for sessions.

you gotta be careful using 2 modules in a pretty difficult area, maybe you should view this magento open source full page cache

Community
  • 1
  • 1
user1743741
  • 134
  • 1
  • 6
0

Have a read of the article I wrote on Full Page Caching in Magento. It highlights a bug fix which suddenly makes the whole cacheing mechanism make sense!

http://www.excitedcroc.com/article/why-the-magento-full-page-cache-doesnt-expire

Essentially there is a bug in the way Magento uses the Zend Framework caching mechanism.

The issue is that the Zend library cache classes and the Magento enterprise cache classes use a mix of null and false in their functions which produce the lifetime value. Because null !== false a default lifetime of 10 days is always used. The problem stems from the processRequestResponse function in app/code/core/Enterprise/PageCache/Model/Processor.php. Because no lifetime value is passed to the cache instance on save it defaults to null.

Changing the default value for the lifetime parameter of the save function of app/code/core/Mage/Core/Model/Cache.php will fix this issue. Just set it to false instead of null ( the article linked above explains fully why ).

- public function save( $data, $id, $tags = array(), $lifeTime = null )

+ public function save( $data, $id, $tags = array(), $lifeTime = false )

Chris Pook
  • 1
  • 1
  • Lone link is considered a poor answer (see [faq#deletion]) since it is meaningless by itself and **target resource is not guaranteed to be alive in the future**. [It would be preferable](http://meta.stackexchange.com/q/8259) to include the essential parts of the answer here, and provide the link for reference. – j0k Feb 08 '13 at 10:56
  • what you say is valuable but totally not related to the question – WonderLand Apr 20 '18 at 09:09