How to get a random element of the filtered queryset efficiently?

Question

I try to pull a random object from the already filtered queryset this way:

Product.objects.filter(active=True).order_by('?').first()

Currently it is solved with .order_by('?') but it seems to be very slow.

I try to implement the solution of this question to my view.

It does not work so I am not sure if I am doing it the right way.

products = Product.objects.filter(
    active=True,
)
count = products.aggregate(count=Count('id'))['count']
random_index = randint(0, count - 1)
product = products[random_index]

This is my code now and it throws the ValueError:

ValueError: low >= high

Please, help me to speed it up.

This is because you have only one active `Product` in the database. — Willem Van Onsem, Jul 14 '21 at 10:24
if your PK is an integer, just grab the min and max then grab a random range using vanilla python, pass that into the query set `Product.objects.filter(active=True, id__in=[rand_range])` — Umar.H, Jul 14 '21 at 10:51
I believe you are using `numpy.random.randint` in which case `high` is exclusive and as @Willem says you have only one active `Product` so `randint(0, 0)` does not make much sense, you simply need to change `count - 1` to `count` (Although your code will work perfectly fine if you just use `random.randint` instead of `numpy.random.randint`, unless of course you have _no_ active products) — Abdul Aziz Barkat, Jul 14 '21 at 11:37

Mojtaba Arezoomand · Answer 1 · 2021-07-14T11:09:18.070

2

you can do this:

import random

products = Product.objects.filter(
    active=True,
)
items = list(products)

# if you want only a single random item
random_item = random.choice(items)

or this one:

from random import randint

count = Product.objects.count()

try:
    random_choice = randint(0, count - 1)
except:
    # if count == 0 then it will raise a ValueError.
    random_choice = 0  

random_object = Product.objects.none()
if random_choice > 0:
    random_object = Product.objects.all()[random_choice]

P.s1: Second solution is more efficient.
P.s2: Don't use order_by('?').

edited Jul 14 '21 at 11:09

answered Jul 14 '21 at 10:43

Mojtaba Arezoomand

2,140
8
23

1

This will load all active records into memory then get a random record from them, which might be problamatic if he has a lot of records – Jameel Hamdan Jul 14 '21 at 10:45
Yeah, there are so many ways to do it. just updated the answer. @JameelHamdan – Mojtaba Arezoomand Jul 14 '21 at 10:49
The second approach will fail if `Product` only has one row – Brian Destura Jul 14 '21 at 10:52
it's just an easy `if else` or a `try except`? @bdbd – Mojtaba Arezoomand Jul 14 '21 at 10:53
Actually I was wrong. Tried `random.randint(0, 0)`, it will always return 0. So your 2nd approach should be fine, but will stil fail when `Product` does not have any rows – Brian Destura Jul 14 '21 at 10:58
The 2nd solution doesn't satisfy the requirement to only choose items that are `active=True`. Also, it's quirky in that you either get an `EmptyQuerySet` or an actual `Product` object. – AKX Jul 14 '21 at 11:25
Suggestion: Change `random_object = Product.objects.none()` to `random_object = None` since getting none or an object is better than getting an empty queryset or an object. – Abdul Aziz Barkat Jul 14 '21 at 11:41

score 2 · Accepted Answer · answered Jul 14 '21 at 11:13

If .order_by('?') is too slow, a decent option (depending on the number of products) might be to query for the PKs of the items, then pick one at random:

qs = Product.objects.filter(active=True)
pks = list(qs.values_list('id', flat=True))
random_pk = random.choice(active_product_pks)
random_item = qs.get(pk=random_pk)

You could cache pks in the Django cache for a while, too, if that suits your business requirements.

How to get a random element of the filtered queryset efficiently?

2 Answers2