146

Is it possible to filter a Django queryset by model property?

i have a method in my model:

@property
def myproperty(self):
    [..]

and now i want to filter by this property like:

MyModel.objects.filter(myproperty=[..])

is this somehow possible?

djvg
  • 11,722
  • 5
  • 72
  • 103
schneck
  • 10,556
  • 11
  • 49
  • 74
  • It is in SQLAlchemy: http://docs.sqlalchemy.org/en/latest/orm/extensions/hybrid.html and you can connect django with SQLAlchemy via https://pypi.python.org/pypi/aldjemy but I'm doubtful that the two could be connected the way you want them to be. – rattray May 03 '16 at 06:28

8 Answers8

120

Nope. Django filters operate at the database level, generating SQL. To filter based on Python properties, you have to load the object into Python to evaluate the property--and at that point, you've already done all the work to load it.

Glenn Maynard
  • 55,829
  • 10
  • 121
  • 131
  • 18
    bad luck that this feature is not implemented, would be an interesting extension to at least filter out matching objects _after_ the resultset has been build. – schneck Jul 30 '09 at 09:24
  • 1
    how to deal with it in admin? Is there some workaround? – andilabs Feb 13 '14 at 14:52
59

I might be misunderstanding your original question, but there is a filter builtin in python.

filtered = filter(myproperty, MyModel.objects)

But it's better to use a list comprehension:

filtered = [x for x in MyModel.objects if x.myproperty()]

or even better, a generator expression:

filtered = (x for x in MyModel.objects if x.myproperty())
Clint
  • 4,255
  • 1
  • 20
  • 19
35

Riffing off @TheGrimmScientist's suggested workaround, you can make these "sql properties" by defining them on the Manager or the QuerySet, and reuse/chain/compose them:

With a Manager:

class CompanyManager(models.Manager):
    def with_chairs_needed(self):
        return self.annotate(chairs_needed=F('num_employees') - F('num_chairs'))

class Company(models.Model):
    # ...
    objects = CompanyManager()

Company.objects.with_chairs_needed().filter(chairs_needed__lt=4)

With a QuerySet:

class CompanyQuerySet(models.QuerySet):
    def many_employees(self, n=50):
        return self.filter(num_employees__gte=n)

    def needs_fewer_chairs_than(self, n=5):
        return self.with_chairs_needed().filter(chairs_needed__lt=n)

    def with_chairs_needed(self):
        return self.annotate(chairs_needed=F('num_employees') - F('num_chairs'))

class Company(models.Model):
    # ...
    objects = CompanyQuerySet.as_manager()

Company.objects.needs_fewer_chairs_than(4).many_employees()

See https://docs.djangoproject.com/en/1.9/topics/db/managers/ for more. Note that I am going off the documentation and have not tested the above.

rattray
  • 5,174
  • 1
  • 33
  • 27
19

Looks like using F() with annotations will be my solution to this.

It's not going to filter by @property, since F talks to the databse before objects are brought into python. But still putting it here as an answer since my reason for wanting filter by property was really wanting to filter objects by the result of simple arithmetic on two different fields.

so, something along the lines of:

companies = Company.objects\
    .annotate(chairs_needed=F('num_employees') - F('num_chairs'))\
    .filter(chairs_needed__lt=4)

rather than defining the property to be:

@property
def chairs_needed(self):
    return self.num_employees - self.num_chairs

then doing a list comprehension across all objects.

TheGrimmScientist
  • 2,812
  • 1
  • 27
  • 25
15

I had the same problem, and I developed this simple solution:

objects = [
    my_object 
    for my_object in MyModel.objects.all() 
    if my_object.myProperty == [...]
]

This is not a performatic solution, it shouldn't be done in tables that contains a large amount of data. This is great for a simple solution or for a personal small project.

Vitalate
  • 196
  • 1
  • 5
  • 3
    Unfortunately this puts quite a heavy load on to the database and memory as you are querying for all objects first and then doing the list comprehension. Once you run it in real production environment with millions of rows, this will quickly start killing your database. – StanKosy Jan 29 '22 at 02:39
3

PLEASE someone correct me, but I guess I have found a solution, at least for my own case.

I want to work on all those elements whose properties are exactly equal to ... whatever.

But I have several models, and this routine should work for all models. And it does:

def selectByProperties(modelType, specify):
    clause = "SELECT * from %s" % modelType._meta.db_table

    if len(specify) > 0:
        clause += " WHERE "
        for field, eqvalue in specify.items():
            clause += "%s = '%s' AND " % (field, eqvalue)
        clause = clause [:-5]  # remove last AND

    print clause
    return modelType.objects.raw(clause)

With this universal subroutine, I can select all those elements which exactly equal my dictionary of 'specify' (propertyname,propertyvalue) combinations.

The first parameter takes a (models.Model),

the second a dictionary like: {"property1" : "77" , "property2" : "12"}

And it creates an SQL statement like

SELECT * from appname_modelname WHERE property1 = '77' AND property2 = '12'

and returns a QuerySet on those elements.

This is a test function:

from myApp.models import myModel

def testSelectByProperties ():

    specify = {"property1" : "77" , "property2" : "12"}
    subset = selectByProperties(myModel, specify)

    nameField = "property0"
    ## checking if that is what I expected:
    for i in subset:
        print i.__dict__[nameField], 
        for j in specify.keys():
             print i.__dict__[j], 
        print 

And? What do you think?

akrueger
  • 365
  • 3
  • 16
  • Generally it seems like a decent work around. I wouldn't say it is ideal, but it beats having to fork a repository to modify the model for packages you installed from PyPI every time you need something like this. – hlongmore Mar 21 '19 at 07:43
  • And now that I've had time to play with it a bit: the real downside to this approach is that querysets returned by .raw() are not full-fledged querysets, by which I mean there are queryset methods that are missing: `AttributeError: 'RawQuerySet' object has no attribute 'values'` – hlongmore Mar 21 '19 at 08:36
  • Take care that building such queries by string concatenation is dangerous. You should use prepared statements somehow. – lbris Feb 20 '22 at 13:55
1

i know it is an old question, but for the sake of those jumping here i think it is useful to read the question below and the relative answer:

How to customize admin filter in Django 1.4

Community
  • 1
  • 1
FSp
  • 1,545
  • 17
  • 37
  • 2
    For those skimming this answer - this link is to information on implementing List Filters in the Django Admin using "SimpleListFilter". Useful, but not an answer to the question except in a very specific case. – jenniwren Nov 09 '18 at 21:43
0

It may also be possible to use queryset annotations that duplicate the property get/set-logic, as suggested e.g. by @rattray and @thegrimmscientist, in conjunction with the property. This could yield something that works both on the Python level and on the database level.

Not sure about the drawbacks, however: see this SO question for an example.

djvg
  • 11,722
  • 5
  • 72
  • 103
  • Your code review question link gives a notice that it was voluntarily removed by its author. Would you mind updating your answer here, either with a link to code, or with an explanation, or just remove your answer? – hlongmore Mar 21 '19 at 07:57
  • @hlongmore: Sorry for that. That question was moved to SO. I fixed the link above. – djvg Mar 21 '19 at 09:21