Convert list into queryset

Question

To improve performance, In my project most of the model instances are stored in cache as list values. But all generic views in Django Rest Framework expect them to be queryset objects. How can I convert the values I got from list into a queryset like objeccts, such that I can use generic views.

Say, I have a function like

def cache_user_articles(user_id):
    key = "articles_{0}".format(user_id)

    articles = cache.get(key)

    if articles is None:
        articles = list(Article.objects.filter(user_id = user_id))
        cache.set(key, articles)

    return articles

In my views.py,

class ArticleViewSet(viewsets.ModelViewSet):

    ...

    def get_queryset(self, request, *args, **kwargs):
        return cache_user_articles(kwargs.get(user_id))

But, this of course this wouldn't work as Django Rest Framework expects the result of get_queryset to be QuerySet object, and on PUT request it would call 'get' method on it. Is there any way, I could make it to work with generic DRF views.

Why do you convert query set to `list` before caching? http://stackoverflow.com/questions/4631865/caching-query-results-in-django — Ozgur Vatansever, Sep 23 '15 at 20:54
before, you could do this https://gist.github.com/zmcghee/1055992 but not anymore, sadly — DevLounge, Sep 23 '15 at 20:54
Why don't you just cache a list but return the queryset in your `cache_user_articles` function? — Shang Wang, Sep 23 '15 at 20:55

score 5 · Accepted Answer · answered Sep 23 '15 at 21:00

That's the place where Python like dynamic languages really shine due to Duck Typing. You could easily write something that quacks like a QuerySet.

import mongoengine
from bson import ObjectId


class DuckTypedQuerySet(list):

    def __init__(self, data, document):
        if not hasattr(data, '__iter__') or isinstance(data, mongoengine.Document):
            raise TypeError("DuckTypedQuerySet requires iterable data")

        super(DuckTypedQuerySet, self).__init__(data)

        self._document = document

    @property
    def objects(self):
        return self

    def _query_match(self, instance, **kwargs):
        is_match = True

        for key, value in kwargs.items():
            attribute = getattr(instance, key, None)

            if isinstance(attribute, ObjectId) and not isinstance(value, ObjectId):
                attribute = str(attribute)

            if not attribute == value:
                is_match = False
                break

        return is_match


    def filter(self, **kwargs):
        data = filter(lambda instance: self._query_match(instance, **kwargs), self)

        return self.__class__(data, self._document)

    def get(self, **kwargs):
        results = self.filter(**kwargs)

        if len(results) > 1:
            raise self._document.MultipleObjectsReturned("{0} items returned, instead of 1".format(len(results)))

        if len(results) < 1:
            raise self._document.DoesNotExist("{0} matching query does not exist.".format(str(self._document)))

        return results[0]

    def first(self):
        return next(iter(self), None)

    def all(self):
        return self

    def count(self):
        return len(self)


def cache_user_articles(user_id):
    key = "articles_{0}".format(user_id)

    articles = cache.get(key)

    if articles is None:
        articles = DuckTypedQuerySet(list(Article.objects.filter(user_id = user_id)), document = Article)
        cache.set(key, articles)

    return articles

Ofcourse, this is not an exhaustive implementation. You might need to add other methods that exists in queryset. But I think these will do for simple use-cases. Now you can make do with generic implementations of Django Rest Framework.

score 0 · Answer 2 · answered Sep 23 '15 at 22:15

What about this? (I have mocked the redis cache with a class variable)

class CachedManager(models.Manager):

    cache = dict()

    def cached(self, user_id):
        cached = self.cache.get(user_id, [])
        if not cached:
            self.cache[user_id] = [article.pk for article in self.filter(user_id=user_id)]

        return self.cache[user_id]


class Article(models.Model):

    objects = CachedManager()

    user_id = models.IntegerField()
    #  Whatever fields your Article model has

Then in your views or wherever you need it:

you can call Article.objects.cached(<a user id>)

Convert list into queryset

2 Answers2