4

I'm trying to use a Memcached instance of AWS ElastiCache with a Django project. It seems to be caching a view for a user, but if you come in on a different PC, it isn't cached until called from that PC (or same PC with different browser).

I'm not sure what I've got wrong.

Within settings.py I have

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': os.environ.get('CACHE_LOCATION','127.0.0.1:11211'),
    }
}

MIDDLEWARE = [
    'core.middleware.DenyIndexMiddleware',
    'core.middleware.XForwardedForMiddleware',
    'core.middleware.PrimaryHostRedirectMiddleware',
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.cache.UpdateCacheMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'django.contrib.redirects.middleware.RedirectFallbackMiddleware',
    'masquerade.middleware.MasqueradeMiddleware',
    'django.middleware.locale.LocaleMiddleware',
    'django.contrib.sites.middleware.CurrentSiteMiddleware',
    'cms.middleware.user.CurrentUserMiddleware',
    'cms.middleware.page.CurrentPageMiddleware',
    'cms.middleware.toolbar.ToolbarMiddleware',
    'cms.middleware.language.LanguageCookieMiddleware',
    'cms.middleware.utils.ApphookReloadMiddleware',
    'django.middleware.cache.FetchFromCacheMiddleware',
]

I've then cached the views using cache_page

path('<str:service_type>/<str:location>/', cache_page(60*60)(views.canonical_search), name="canonical-search"),

How do I cache the site so that the page is cached irrespective of the user?

EDIT I've noticed that it never caches when using the user is logged in.

HenryM
  • 5,557
  • 7
  • 49
  • 105
  • Hello! Is the URL the same for all users? I suspect it is, but if it's not, there lies the problem, since the cache system caches on a per-URL basis instead of on a per-view basis. – Pentracchiano Jun 02 '20 at 14:27
  • url is the same for all users – HenryM Jun 02 '20 at 14:36
  • By user, do you mean logged-in/anonymous? Is there any cache policy in your web server (nginx/apache)? – SebCorbin Jun 02 '20 at 15:13
  • @SebCorbin I'm using ElasticBeanstalk and I've not set any cache explicitly. I don't know if ElasticBeanstalk has – HenryM Jun 02 '20 at 16:39

3 Answers3

5

Watch out for the Vary header, that cache_page() takes into account.

Usually, some middlewares may add a Vary header, for example :

  • CsrfViewMiddleware adds Cookie,
  • GZipMiddlewareadds Accept-Encoding
  • LanguageCookieMiddleware may add Accept-Language

meaning that as soon as you have a different Cookie (session), encoding, or language, you have a different version of cache for your page.

As for you case, the CsrfViewMiddleware may be the problem, you can add the decorator @csrf_exempt to your view so that the Vary: Cookie header is not set in the response.

More info at https://docs.djangoproject.com/en/3.0/topics/cache/#using-vary-headers

SebCorbin
  • 1,645
  • 1
  • 12
  • 23
  • Then I suggest you inspect the requests and responses both browsers send and receive, to compare `Vary` header of the response and the headers sent by the browsers. – SebCorbin Jun 02 '20 at 16:41
  • The `Vary` header in both says `vary: Accept-Language,Cookie,X-Forwarded-Proto,Accept-Encoding,User-Agent` I don't understand what this is telling me – HenryM Jun 02 '20 at 17:13
  • So yeah, you have a lot a possible variations, I suggest you search for `Vary` or `Cookie` in the middlewares you are using and try to debug which one is adding it. Also `User-Agent` is surely a probable cause for cache miss. – SebCorbin Jun 03 '20 at 14:20
  • What do I do once I've found which one is varying it? – HenryM Jun 04 '20 at 10:09
  • How do I tell it not to vary on headers? – HenryM Jun 04 '20 at 10:14
  • Well you have to read the documentation about this middleware and see if there is a flag to cancel/understand with it's setting the `Vary ` value. As a last resort, you can write your own middleware, place it after in the list, and remove the value from `Vary` but it would advise against it. – SebCorbin Jun 04 '20 at 13:34
2

Though Django documentation, you can read this: Django’s cache framework

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': '127.0.0.1:11211',
    }
}

If you need to store a page in the local 

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
        'LOCATION': 'c:/foo/bar',
    }
}

But, my recommendation is store the results (database result, assets, etc), as @PVSK show in this thread:

from django.core.cache import cache    
def sample(request):
        cached_data = cache.get_many(['query1', 'query2'])
        if cached_data:
            return render(request, 'sample.html', {'query1': cached_data['query1'], 'query2': cached_data['query2']})
        else:
            queryset1 = Model.objects.all()
            queryset2 = Model2.objects.all()
            cache.set_many({'query1': queryset1 , 'query2': queryset2 }, None)
            return render(request, 'sample.html', {'query1': queryset1 , 'query2': queryset2})
Benjamin RD
  • 11,516
  • 14
  • 87
  • 157
0

Hm, at first I wondered if you're running into some cache default limitations. You are not using OPTIONS in your CACHE backend definition, so the cache is limited to 300 entries per default.

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': os.environ.get('CACHE_LOCATION','127.0.0.1:11211'),
        'OPTIONS': {
            'MAX_ENTRIES': 1000
        }

    }
}

The next possible problem, that we also had, is that the cache_key generation takes the full QUERY_STRING into account, so (the ?param=bla). But you already stated that the url is the same for all users.

Next up, as SebCorbin correctly identified, are the potential Vary issues.

UpdateCacheMiddleware will never cache a cookie-setting response to a cookie-less request.

    def process_response(self, request, response):
        #...

        # Don't cache responses that set a user-specific (and maybe security
        # sensitive) cookie in response to a cookie-less request.
        if not request.COOKIES and response.cookies and has_vary_header(response, 'Cookie'):
            return response

        # ...

The order of execution of middlewares is top-to-bottom for process_request and bottom-to-top for process_response.

I suspect that one of lower middlewares (or the view) is setting a cookie, so you might be able to work around this issue by moving the 'django.middleware.cache.UpdateCacheMiddleware' below the offending middleware, but you risk loosing features if you don't move feature-middlewares like LocaleMiddleware as well.

If your view code is setting cookies you'll need to switch to the low-level cache API to cache costly operations (or move the cookie logic into a middleware that lives above the UpdateCacheMiddleware middleware).

ACimander
  • 1,852
  • 13
  • 17