How to merge multiple query sets in DJANGO

Question

I have to merge the querysets below in a single list:

result_list_1 = Col.objects.defer("tags").filter(producer__username__icontains=crit) 
result_list_2 = Col.objects.defer("tags").filter(name__icontains=crit)
result_list_3 = Col.objects.defer("tags").filter(description__icontains=crit)
result_list_4 = Col.objects.filter(tags__name__icontains=crit)
...

Each result_list contains items, which have a unique numeric id I can use to make sure there are no dups.

I can't use | while querying the DB or Q objects.

How do I merge the resulsets in one single list?

See here: http://stackoverflow.com/questions/431628/how-to-combine-2-or-more-querysets-in-a-django-view — Alex, May 19 '14 at 22:49
Thanks Alex, I saw that one. There is a reference in a comment about merging elements without duplicates, but it is not very clear. — Miki, May 19 '14 at 22:58
But why? If you give us some context to your problem it might be easier to provide you with a solution. — Steinar Lima, May 19 '14 at 23:00
The code sample above shows only 4 querysets, but I actually have a lot more, some of them with way more complex filtering criteria. They cannot be combined. I have to merge the resulting querysets. — Miki, May 19 '14 at 23:09
What about bitwise operations on the *querysets*? (see [this answer](http://stackoverflow.com/a/796381/835002)) E.g: `result_list_1 | result_list_2` etc. — Steinar Lima, May 19 '14 at 23:29
What you call "bitwise operations" works perfect. There should be a different name to call that though. If you post it as an answer I'll accept it. — Miki, May 20 '14 at 13:22

steffens21 · Answer 1 · 2014-05-20T00:03:49.693

5

What about a slight modification of itertools.chain that makes sure you don't get dupes:

def unique_chain(*iterables):
    known_ids = set()
    for it in iterables:
        for element in it:
            if element.id not in known_ids:
                known_ids.add(element.id)
                yield element

With that you can create your combined list:

combined_list = list(unique_chain(result_list_1, result_list_2, ... ))

edited May 20 '14 at 00:03

answered May 19 '14 at 23:57

steffens21

699
6
12

1

Thanks Steffens. That did the trick. I suspect there might be more efficient methods but this one works. – Miki May 20 '14 at 00:14
I hate to rely on this technique but for quick prototyping, it saved my day. Now I just need to find an efficient applicable solution. Yeah its time complexity that takes my time But thank you – Willy satrio nugroho Feb 20 '22 at 04:05

score 3 · Answer 2 · answered May 19 '14 at 23:46

Then you can't have a QuerySet and it's django-related functionality, but you can use itertools.chain which is specifically for merging multiple iterables.

import itertools

merged = itertools.chain(qs1, qs2, qs3, qs4)

for element in merged:
    print(element)

How to merge multiple query sets in DJANGO

2 Answers2