Manually combine multiple Django QuerySets, and serialize result using DRF

Question

I have been researching this for a couple days. Unfortunately, all of the proposed solutions I have found so far don't exactly work for me.

I am looking to manually combine two Django QuerySets into a single Django model, which I then want to serialize using a Django Rest Framework serializer. I then use the serializer for outputting JSON.

I have found various solutions on SO suggesting the use of itertools and chain, but it is unclear then how to serialize the result of chain.

Note that my goal here is for web application performance purposes. Each QuerySet works fine independently, but it requires two separate Ajax calls to retrieve the results. I would prefer to make only one Ajax call, manually combine the results on the server-side, then return the combined JSON.

This SO answer is close, but I can't figure out how to serialize the result of a chain, using DRF serializers.

score 11 · Answer 1 · answered Sep 19 '16 at 20:21

After a couple more days of Googling for answers to this problem, I finally stumbled on this page which explains how to do pretty much what I wanted to do. I have posted my own answer here in case someone else has the same problem.

The technique seems simple enough now that I understand it, but I had a hard time finding anything online describing this technique. Basically, you chain together multiple querysets, then convert the chain to a Python list. Then iterate the list, manually calling the correct serializer, and appending to a dictionary. Then you can dump the dictionary to JSON.

I have posted the relevant code in case that link goes dead:

    def get_queryset(self):
        queryset_a = Post.objects.all()
        queryset_b = FriendRequest.objects.filter(is_unanswered=True)

        # Create an iterator for the querysets and turn it into a list.
        results_list = list(chain(queryset_a, queryset_b))

        # Optionally filter based on date, score, etc.
        sorted_list = sorted(results_list, key=lambda instance: -instance.date_created)

        # Build the list with items based on the FeedItemSerializer fields
        results = list()
        for entry in sorted_list:
            item_type = entry.__class__.__name__.lower()
            if isinstance(entry, Post):
                serializer = PostSerializer(entry)
            if isinstance(entry, FriendRequest):
                serializer = FriendRequestSerializer(entry)

            results.append({'item_type': item_type, 'data': serializer.data})

        return results

For anyone else who had this issue that I did on the ```sorted_list``` line. If your ```instance.date_created``` field is a ```datetime.datetime``` object, django would raise a TypeError. This is because you cannot negate a datetime.datetime object, only a number. To fix it, convert it to a numerical value by: using ```instance.date_created.timestamp()``` instead of ```instance.date_created``` — Caleb, Apr 08 '23 at 00:51

score 4 · Answer 2 · answered Sep 16 '16 at 18:08

4

Are the two querysets from the same model? If so, you can just say:

qs1 = Potato.objects.filter(size = PotatoSize.large)
qs2 = Potato.objects.filter(color = VegetableColor.brown)
combined_qs = qs1 | qs2

If they are from different models, but the models are related to each other via inheritance (i.e. one is an abstract subclass) you can still do this:

qs1 = RootVegetable.objects.filter(color = VegetableColor.brown)
qs2 = Potato.objects.filter(size = PotatoSize.large)

qs2.model = RootVegetable

# this will return a QS of RootVegetables
combined_qs = qs1 | qs2

You can then serialize the combined queryset.

answered Sep 16 '16 at 18:08

user2030378

155
3
9

The models are related, but one is not an abstract subclass of the other. Basically, I need a way to attach two QuerySets to a single model, then serialize the whole thing. And that single model is an unmanaged model (i.e. there is no database table corresponding). Basically I'm trying to group several related sets of data together. So unfortunately, that suggestion doesn't really help too much. – John Cleveland Sep 16 '16 at 23:48
I added a link to an SO answer that is close. It just doesn't explain how to serialize that result, especially when the model I'm serializing isn't managed. – John Cleveland Sep 16 '16 at 23:57

Manually combine multiple Django QuerySets, and serialize result using DRF

2 Answers2