Django delete cache with specific key_prefix

Question

I'm using Django's per-view @cache_page decorator and have set a different key_prefix for each view.

I've previously deleted the cache with:

from django.core.cache import cache
cache.clear()

But what if I just want to delete the keys containing a specific key_prefix? I can obviously do it by just connecting to the database and delete with raw sql but I wonder if it can be done with 'pure' Django?

I'm using a database cache, not a memory cache.

I'm using Django 1.11 and Python 3.6

Answer 1

I achieved what I want with the code like this:

# first: pip install django-redis

from django.core.cache import cache

cache.delete_many(keys=cache.keys('*.letters.*'))

It deletes all caches which keys contain "letters".

EDIT: I use redis server. I didn't test it for other cache servers.

Answer 2

As @e4c5 mentioned cache is used for fast stuff, you should be using redis for the same. But since your question is about database I would answer the same.

There is no existing function to do this in Django. But then best part of python is you can easily monkey path to add new functionality. Below is a test request I created

def index(request):
    cache.set("name", "tarun")
    cache.set("name_1", "tarun")
    cache.set("name2", "tarun")
    cache.set("name_4", "tarun")
    cache.set("nam", "tarun")

    cache.clear(prefix="name")
    nam = cache.get("nam")
    name_4 = cache.get("name_4", default="deleted")
    return HttpResponse("Hello, world. nam={nam}, name_4={name_4}".format(nam=nam, name_4=name_4))

To get the prefix functionality you need to add below patch code in some place. I used settings.py as such

original_clear = None


def patch_clear():
    from django.db import connections, router
    from django.core.cache.backends.db import DatabaseCache

    def __clear(self, prefix=None, version=None):
        db = router.db_for_write(self.cache_model_class)
        connection = connections[db]
        table = connection.ops.quote_name(self._table)
        with connection.cursor() as cursor:
            if prefix is None:
                cursor.execute('DELETE FROM %s ' % table)
            else:
                prefix = self.make_key(prefix, version)
                cursor.execute("DELETE FROM %s where cache_key like '%s%%'" % (table, prefix))

    global original_clear
    original_clear = DatabaseCache.clear
    DatabaseCache.clear = __clear

patch_clear()

Answer 3

TLDR; cache.delete and cache.delete_many are your available options.

Long answer. @cache_page is over rated. When you use this decorator, you often find that the cache always contains many more cache entries than you expected. You end up wanting to delete a whole bunch of cache entries. Which seems to be exactly what has happened here.

I'm using a database cache, not a memory cache.

One of the main ideas of using caching is to reduce the load on the server another is to reduce expensive calculations or db queries. But in reality a great many web pages do not have expensive calculations. Most slow queries can be optimized by carefully choosing your indexes.

If the database itself is that cache, you are not reducing the load on the database. And what if you need to display different content for different users? This get's awfully complicated.

what if I just want to delete the keys containing a specific key_prefix?

Consider using redis. This is one of the best caching backends available in django (as a third party module). Being able to delete multiple keys in a single command is one of the many useful features of redis.

Answer 4

With LocMemCache, you can delete all the cache values whose keys contain name as shown below. *list() is needed for cache._cache.keys() otherwise you get error and specifying a version is needed for cache.delete() otherwise you cannot delete all the cache values whose keys contain name and the answer of my question explains the default version of a cache value with LocMemCache:

from django.core.cache import cache
from django.http import HttpResponse

def test(request):
    cache.set("first_name", "John")
    cache.set("last_name", "Smith", version=2)
    cache.set("age", 36, version=3)
    cache.set("gender", "Male")
    
    print(cache._cache.keys())
    # odict_keys([':1:gender', ':3:age', ':2:last_name', ':1:first_name'])

            # `list()` is needed
    for key in list(cache._cache.keys()):
        new_key = key.split(":", 2)[2]
        version = key.split(":", 2)[1]

        if "name" in new_key:
            cache.delete(new_key, version=version)

    print(cache._cache.keys())
    # odict_keys([':1:gender', ':3:age'])

    return HttpResponse("Test")