11

I would like to know if I can get in a single query, All the objects of certain model where its date's year equals the year of the Max('date') of the model. For example, using the models from the Aggregation Django Docs, how can I get All the Books published in the year of the more recently published Book?

All the examples in the docs filter by immediate values (pubdate__year=2006), but I need to use a calculated value over the same object in the same query.

Of course, I could do this by performing two queries: one for getting the max year, and a second one to filter by that year, but I think it should be possible to do it in a single query. It's just I haven't figured it out yet.

Thanks for your help!

EDIT:

Since some of you have given similar answers, I'm writing this update so my problem can be better understood.

This is my model:

class Expo(models.Model):

    class Meta:
        verbose_name= _('Expo')
        verbose_name_plural = _('Expos')

    name = models.CharField(max_length=255)
    place = models.CharField(max_length=255, null=True, blank=True)
    date = models.DateField()
    bio = models.ForeignKey(Bio, related_name='expos')

I need "All the Expos that happened in the latest year of the list of Expos stored in my database"

To resolve this, I'm doing this:

from django.db.models import Max
max_year = Expo.objects.all().aggregate(Max('date'))['date__max'].year
expos = Expo.objects.filter(date__year=max_year)

But this, I understand that performs two queries on the database. I would like an expression that let me get the same result, but performing a single query.

I've tried as suggested:

Expo.objects.annotate(max_year=Max('date__year')).filter(date__year=F('max_year'))

But get the error:

FieldError: Join on field 'date' not permitted. Did you misspell 'year' for the lookup type?

I also have tried:

Expo.objects.annotate(max_date=Max('date')).filter(date__year__gte=F('max_date__year'))

but I get the error:

FieldError: Cannot resolve keyword 'max_date' into field. Choices are: bio, date, id, items, name, place, max_date

Notice that it says that it can't resolve 'max_date', but it appears listed among the choices. Weird.

Again, Thanks a lot for your help! :)

Community
  • 1
  • 1
Throoze
  • 3,988
  • 8
  • 45
  • 67
  • What have you tried so far? In case you have problems finding the docs, here's an example of pretty much exactly what you are trying to do: https://docs.djangoproject.com/en/dev/topics/db/aggregation/#filtering-on-annotations – Wolph Feb 12 '14 at 09:31
  • It should be noted that you have to use it in combination with the `F()` expression: https://docs.djangoproject.com/en/dev/topics/db/queries/#filters-can-reference-fields-on-the-model – Wolph Feb 12 '14 at 09:33
  • Thanks! The `F()` object seems to be what I need, but the bigger problem is that I need to compare the years, not the dates. I tried as suggested in the answer below: `Expo.objects.annotate(max_date=Max('date')).filter(date__gte=F('max_date'))` (note the `__gte`) but it returned not expected instances (with dates before the max date). I also tried `Expo.objects.annotate(max_date=Max('date')).filter(date__year__gte=F('max_date__year'))`, but got this bizarre error: `FieldError: Cannot resolve keyword 'max_date' into field. Choices are: bio, date, id, items, name, place, max_date` – Throoze Feb 12 '14 at 10:30
  • Performing statement in a **single query don't will improve performance**. Also, remember than, instead `Max` agregation you can get first model sorting by `-date`. For me it is more clear, readable and fast splitting code in two sentences. If you don't believe it, try to write your query in a single SQL brand agnostic sentence. – dani herrera Feb 12 '14 at 13:55
  • @danihp: Could you please explain me why it won't improve performance? That is precisely what worries me. Thanks! – Throoze Feb 12 '14 at 14:05
  • Throoze, answered in my own answer. I hope it is useful. – dani herrera Feb 12 '14 at 14:28

3 Answers3

10

Performing statement in a single query is no guarantee to improve performance, this is easy to understand if you try to write an agnostic RDBMS brand SQL single sentence for yours requirements. Also, you lost in readability.

In my opinion, you can see and elegant solution in this approach:

  1. Get last Expo by date .
  2. Do a simple filter query.

For your code:

max_year = Expo.objects.latest('date').date.year
expos = Expo.objects.filter(date__year=max_year)

Remember you can cache max_year, also create a DESC index over date can helps.

Community
  • 1
  • 1
dani herrera
  • 48,760
  • 8
  • 117
  • 177
  • Thanks! this was what I was concerned about when asked the question. That thing with the cache and the context processor, is also a very good idea. Thanks! – Throoze Feb 12 '14 at 14:48
2

Here is how you can do something using a combination of Annotation and F object

To filter on Max date:

ModelClass.objects.annotate(max_date=Max('pubdate')).filter(pubdate=F('max_date'))

To filter on the year of Max date:

max_date = ModelClass.objects.latest("pubdate").pubdate
published_objs = ModelClass.objects.filter(pubdate__year=max_date.year)

There does not seem to be a way to filter on Max(date.year) in a single query. And as mentioned by @danihp, even a single query is not a guarantee of performance.

Sunny Nanda
  • 2,362
  • 1
  • 16
  • 10
  • hi! after testing your suggestion it doesn't seem to return what I'm expecting. First, what you're suggesting should return the objects whose fields `date` are equal to the `max_date`, and I need the objects whose fields `date`'s year are equal to the `max_date` year. Second, when I tested it in the `manage.py shell`, it returned (i think) all the objects, ignoring the filter. Did you test it? did it work for you? thanks for your help! – Throoze Feb 12 '14 at 10:23
  • Updated the answer to filter on the year as well. The answer is based on my understanding of django (Untested because I don't have a setup similar to yours) – Sunny Nanda Feb 12 '14 at 12:20
  • 1
    This will raise: `FieldError: Join on field 'pubdate' not permitted. Did you misspell 'year' for the lookup type?` on django 1.6. Please, test before post. – dani herrera Feb 12 '14 at 12:55
  • Please, fix your answer and notice me in order to remove my down vote. – dani herrera Feb 12 '14 at 13:56
  • @danihp I took a reference from [django 1.6 docs](https://docs.djangoproject.com/en/1.6/ref/models/querysets/#year). Can you please let me know the mistake, as I still don't understand the reason of the error. – Sunny Nanda Feb 12 '14 at 14:01
  • `__year`, as appears in your link, is under [Field lookups section](https://docs.djangoproject.com/en/1.6/ref/models/querysets/#field-lookups): *"Field lookups are how you specify the meat of an SQL WHERE clause. They’re specified as keyword arguments to the QuerySet methods filter(), exclude() and get()."* ;) – dani herrera Feb 12 '14 at 14:11
  • @danihp Edited the answer accordingly. Thanks for correcting. – Sunny Nanda Feb 12 '14 at 15:42
  • Do we have any function for old date? How do I find old date row? – Piyush S. Wanare Jun 08 '17 at 07:29
2

Throoze try this...

 queryset=Expo.objects.annotate(max_date=Max('date'))
 queryset1=queryset.values().filter(date__gte=F('max_date'))
Adiyat Mubarak
  • 10,279
  • 4
  • 34
  • 50
pramod24
  • 1,096
  • 4
  • 17
  • 38
  • Hi! It should have parenthesis after `values()`, and for some strange reason it also returns values with dates lesser than the max_date. And, as commented above, if I try to compare the years (which is what I need) , I keep getting this error: `FieldError: Cannot resolve keyword 'max_date' into field. Choices are: bio, date, id, items, name, place, max_date`. Thanks anyways for your help :) – Throoze Feb 12 '14 at 11:58
  • Do we have any function for old date? – Piyush S. Wanare Jun 08 '17 at 07:29