Display objects from different models at the same page according to their published date

Question

I have three different models for my app. All are working as I expected.

class Tender(models.Model):
    title = models.CharField(max_length=256)
    description = models.TextField()
    department = models.CharField(max_length=50)
    address = models.CharField(max_length=50)
    nature_of_work = models.CharField(choices=WORK_NATURE, max_length=1)
    period_of_completion = models.DateField()
    pubdat = models.DateTimeField(default=timezone.now)    

class Job(models.Model):
    user = models.ForeignKey(settings.AUTH_USER_MODEL)
    title = models.CharField(max_length=256)
    qualification = models.CharField(max_length=256)
    interview_type = models.CharField(max_length=2, choices=INTERVIEW_TYPE)
    type_of_job = models.CharField(max_length=1, choices=JOB_TYPE)
    number_of_vacancies = models.IntegerField()
    employer = models.CharField(max_length=50)
    salary = models.IntegerField()    
    pubdat = models.DateTimeField(default=timezone.now)

class News(models.Model):
    user = models.ForeignKey(settings.AUTH_USER_MODEL)
    title = models.CharField(max_length=150)
    body = models.TextField()
    pubdat = models.DateTimeField(default=timezone.now)

Now I am displaying each of them at separate page for each of the model (e.g. in the jobs page, I am displaying only the jobs.). But now at the home page, I want to display these according to their published date at the same page. How can I display different objects from different models at the same page? Do I make a separate model e.g. class Post and then use signal to create a new post whenever a new object is created from Tender, or Job, or News? I really hope there is a better way to achieve this. Or do I use multi-table inheritance? Please help me. Thank you.

Update:

I don't want to show each of the model objects separately at the same page. But like feeds of facebook or any other social media. Suppose in fb, any post (be it an image, status, share) are all displayed together within the home page. Likewise in my case, suppose a new Job object was created, and after that a new News object is created. Then, I want to show the News object first, and then the Job object, and so on.

Answer 1

The following should do want you need. But to improve performance you can create an extra type field in each of your models so the annotation can be avoided.

Your view will look something like:

from django.db.models import CharField

def home(request):
    # annotate a type for each model to be used in the template
    tenders = Tender.object.all().annotate(type=Value('tender', CharField()))
    jobs = Job.object.all().annotate(type=Value('job', CharField()))
    news = News.object.all().annotate(type=Value('news', CharField()))

    all_items = list(tenders) + list(jobs) + list(news)

    # all items sorted by publication date. Most recent first
    all_items_feed = sorted(all_items, key=lambda obj: obj.pubdat)

    return render(request, 'home.html', {'all_items_feed': all_items_feed})

---

In your template, items come in the order they were sorted (by recency), and you can apply the appropriate html and styling for each item by distinguishing with the item type:

# home.html
{% for item in all_items_feed %}
    {% if item.type == 'tender' %}
    {% comment "html block for tender items" %}{% endcomment %}

    {% elif item.type == 'news' %}
    {% comment "html block for news items" %}{% endcomment %}

    {% else %}
    {% comment "html block for job items" %}{% endcomment %}

    {% endif %}
{% endfor %}

---

You may avoid the annotation altogether by using the __class__ attribute of the model objects to distinguish and put them in the appropriate html block.

For a Tender object, item.__class__ will be app_name.models.Tender where app_name is the name of the Django application containing the model.

So without using annotations in your home view, your template will look:

{% for item in all_items_feed %}
    {% if item.__class__ == 'app_name.models.Tender' %}

    {% elif item.__class__ == 'app_name.models.News' %}
     ...
    {% endif %}
{% endfor %}

With this, you save extra overhead on the annotations or having to modify your models.

Answer 2

A working solution

There are two working solutions two other answers. Both those involve three queries. And you are querying the entire table with .all(). The results of these queries combined together into a single list. If each of your tables has about 10k records, this is going to put enormous strain on both your wsgi server and your database. Even if each table has only 100 records each, you are needlessly looping 300 times in your view. In short slow response.

An efficient working solution.

Multi table inheritance is definitely the right way to go if you want a solution that is efficient. Your models might look like this:

class Post(models.Model):
    title = models.CharField(max_length=256)
    description = models.TextField()
    pubdat = models.DateTimeField(default=timezone.now, db_index = True)    

class Tender(Post):
    department = models.CharField(max_length=50)
    address = models.CharField(max_length=50)
    nature_of_work = models.CharField(choices=WORK_NATURE, max_length=1)
    period_of_completion = models.DateField()

class Job(Post):
    user = models.ForeignKey(settings.AUTH_USER_MODEL)
    qualification = models.CharField(max_length=256)
    interview_type = models.CharField(max_length=2, choices=INTERVIEW_TYPE)
    type_of_job = models.CharField(max_length=1, choices=JOB_TYPE)
    number_of_vacancies = models.IntegerField()
    employer = models.CharField(max_length=50)
    salary = models.IntegerField()    


class News(models.Model):
    user = models.ForeignKey(settings.AUTH_USER_MODEL)

    def _get_body(self):
        return self.description

    body = property(_get_body)

now your query is simply

 Post.objects.select_related(
   'job','tender','news').all().order_by('-pubdat')   # you really should slice

The pubdat field is now indexed (refer the new Post model I posted). That makes the query really fast. There is no iteration through all the records in python.

How do you find out which is which in the template? With something like this.

{% if post.tender %}

{% else %} 
   {% if post.news %}
   {% else %}
{% else %}

Further Optimization

There is some room in your design to normalize the database. For example it's likely that the same company may post multiple jobs or tenders. As such a company model might come in usefull.

An even more efficient solution.

How about one without multi table inheritance or multiple database queries? How about a solution where you could even eliminate the overhead of rendering each individual item?

That comes with the courtesy of redis sorted sets. Each time you save a Post, Job or News, object you add it to a redis sorted set.

from django.db.models.signals import pre_delete, post_save
from django.forms.models import model_to_dict

@receiver(post_save, sender=News)
@receiver(post_save, sender=Post)
@receiver(post_save, sender=Job)

def add_to_redis(sender, instance, **kwargs):
    rdb = redis.Redis()

    #instead of adding the instance, you can consider adding the 
    #rendered HTML, that ought to save you a few more CPU cycles.

    rdb.zadd(key, instance.pubdat, model_to_dict(instance)

    if (rdb.zcard > 100) : # choose a suitable number
         rdb.zremrangebyrank(key, 0, 100)

Similarly, you need to add a pre_delete to remove them from redis

The clear advantage of this method is that you don't need any database queries at all and your models continue to be really simple + you get catching thrown in the mix. If you are on twitter your timeline is probably generated through a mechanism similar to this.

Answer 3

A straight forward way is to use chain in combination with sorted:

View

# your_app/views.py
from django.shortcuts import render
from itertools import chain
from models import Tender, Job, News

def feed(request):

    object_list = sorted(chain(
        Tender.objects.all(),
        Job.objects.all(),
        News.objects.all()
    ), key=lambda obj: obj.pubdat)

    return render(request, 'feed.html', {'feed': object_list})

Please note - the querysets mentioned above using .all() should be understood as placeholder. As with a lot of entries this could be a performance issue. The example code would evaluate the querysets first and then sort them. Up to some hundreds of records it likely will not have a (measurable) impact on performance - but in a situation with millions/billions of entries it is worth looking at.

To take a slice before sorting use something like:

Tender.objects.all()[:20]

or use a custom Manager for your models to off-load the logic.

class JobManager(models.Manager):
    def featured(self):
        return self.get_query_set().filter(featured=True)

Then you can use something like:

Job.objects.featured()

Template

If you need additional logic depending the object class, create a simple template tag:

#templatetags/ctype_tags.py
from django import template
register = template.Library()

@register.filter
def ctype(value):
    return value.__class__.__name__.lower()

and

#templates/feed.html
{% load ctype_tags %}

<div>
    {% for item in feed reversed %}
    <p>{{ item|ctype }} - {{ item.title }} - {{ item.pubdat }}</p>
    {% endfor %}
</div>

Bonus - combine objects with different field names

Sometimes it can be required to create these kind of feeds with existing/3rd party models. In that case you don't have the same fieldname for all models to sort by.

DATE_FIELD_MAPPING = {
    Tender: 'pubdat',
    Job: 'publish_date',
    News: 'created',
}

def date_key_mapping(obj):
    return getattr(obj, DATE_FIELD_MAPPING[type(obj)])


def feed(request):
    object_list = sorted(chain(
        Tender.objects.all(),
        Job.objects.all(),
        News.objects.all()
    ), key=date_key_mapping)

Answer 4

Do I make a separate model e.g. class Post and then use signal to create a new post whenever a new object is created from Tender, or Job, or News? I really hope there is a better way to achieve this. Or do I use multi-table inheritance?

I don't want to show each of the model objects separately at the same page. But like feeds of facebook or any other social media.

I personally don't see anything wrong using another model, IMHO its even preferable to use another model, specially when there is an app for that.

Why? Because I would never want to rewrite code for something which can be achieved by extending my current code. You are over-engineering this problem, and if not now, you are gonna suffer later.

Answer 5

An alternative solution would be to use Django haystack:

• http://haystacksearch.org/
• http://django-haystack.readthedocs.io/en/v2.4.1/

It allows you to search through unrelated models. It's more work than the other solutions but it's efficient (1 fast query) and you'll be able to easily filter your listing too.

---

In your case, you will want to define pubdate in all the search indexes.

Answer 6

I cannot test it right now, but you should create a model like:

class Post(models.Model):

    pubdat = models.DateTimeField(default=timezone.now)
    tender = models.ForeignKey('Tender')
    job = models.ForeignKey('Job')
    news = models.ForeignKey('News')

Then, each time a new model is created, you create a Post as well and relate it to the Tender/Job/News. You should relate each post to only one of the three models.

Create a serializer for Post with indented serializers for Tender, Job and News.

Sorry for the short answer. If you think it can work for your problem, i'll write more later.