11

This question is about model inheritance in Django.

Almost everything I have read (including the Django documentation itself) strongly recommends doing 'abstract base class' inheritance rather than 'multi-table' inheritance. I agree with the reasoning and as a result am fully behind the recommendation. However, Django does not appear to have support for:

  • polymorphic querying, or
  • model linking (i.e. I cannot create a ForeignKey field to the abstract base class from another model).

Situation

For example, I have some models that implement the 'abstract base class' inheritance pattern:

class Tool(models.Model):
    name = models.CharField(max_length=30)
    group = models.ManyToManyField(ToolGroup, blank=True) # Link to 'ToolGroup' MUST be placed on abstract class
    attributes = models.ManyToManyField(ToolAttributeValue, blank=True)  # Link to 'ToolAttributeValue' MUST be placed on abstract class

    class Meta:
        abstract = True # Almost everything I read strongly recommends against making this its own table


class HandheldTool(Tool):
    electrical_safe = models.BooleanField(default=False)


class PowerTool(Tool):
    compliance_expiry_date = models.DateTimeField()


class ConsumableTool(Tool):
    combustible = models.BooleanField(default=False)
    best_before = models.DateTimeField(null=True)

I also have some grouping and information classes related to tools in general:

# Grouping related structures
#
# ToolHierarchy  >       ToolGroup (n times)       > Tool
# 
#   "Tool Boxes" > "Day Shift"   > "Builders"      > HandheldTool[Hammer]
#                                                  > HandheldTool[Screwdriver - SAFE]
#                                                  > PowerTool[Drill]
#
#                                > "Demo Team"     > HandheldTool[Sledgehammer 1]
#                                                  > PowerTool[Jackhammer]
#                                                  > ConsumableTool[Dynamite]
#
#                > "Night Shift" > "Rock Breakers" > HandheldTool[Hammer]
#                                                  > HandheldTool[Sledgehammer 2]
#                                                  > PowerTool[Rocksaw]

class ToolHierarchy(models.Model):
    name = models.CharField(blank=True, max_length=30)


class ToolGroup(models.Model):
    name = models.CharField(blank=True, max_length=30)
    parent = models.ForeignKey('self', related_name='children', null=True, blank=True)
    hierarchy = models.ForeignKey(ToolHierarchy, null=True, blank=True, related_name='top_level_tools')
    # tools = models.ManyToManyField(Tool) # CANNOT MAKE LINK, as 'Tool' is abstract


# 'Extra-info' structures
#
# ToolAttribute > ToolAttributeValue > Tool
# 
#  'Brand'      > 'Stanley'          > HandheldTool[Hammer]
#                                    > HandheldTool[Sledgehammer 1]
#               > 'ACME'             > HandheldTool[Sledgehammer 2]
#                                    > ConsumableTool[Dynamite]
#
#  'Supplier'   > 'Bash Brothers'    > HandheldTool[Hammer]
#                                    > HandheldTool[Sledgehammer 1]
#                                    > HandheldTool[Sledgehammer 2]
class ToolAttribute(models.Model):
    name = models.CharField(max_length=30)
    data_type = models.CharField(max_length=30) # e.g. "STRING", "INT", "DATE" "FLOAT" -- Actually done with enum
    unit_of_measure = models.CharField(max_length=30, blank=True)


class ToolAttributeValue(models.Model):
    attribute = models.ForeignKey(ToolAttribute)
    value = models.CharField(blank=True, max_length=30)
    # tool = models.ForeignKey(Tool)  # CANNOT MAKE LINK, as 'Tool' is abstract

Problem

Ideally this inheritance model would be implemented via a polymorphic relationship, however the Django ORM does not support it. This is possible with SQLAlchemy and other ORMs like Hibernate.

With the Django ORM, because the Tool class is abstract I cannot create the links like:

  • ToolAttributeValue.tool -> tool_obj or
  • ToolGroup.tools -> [tool_obj_1, tool_obj_2].

Instead I am forced to create the inverse link on the abstract class, despite it modelling a slightly different thing! This then results in all sorts of ugliness on the ToolAttributeValue and ToolGroup objects, which then no longer have a .tools attribute but instead have RelatedManager fields for each subtype. i.e.:

tool_group_obj.handheldtool_set.all()
tool_group_obj.powertool_set.all()
...etc, for every subtype of Tool

Which pretty much destroys the usefulness of the abstract class.

Questions

So, with this in mind, my questions are:

  1. Is this a good case for 'multi-table' inheritance?
  2. Am I trying too hard to force the type inheritance? Should I just get rid of Tool? If yes, then do I have to create a *ToolGroup model for each subclass?
  3. What is the current (Django 1.8) accepted way around this? Surely I am not the first person to build a relational system in Django ;-) And the fact that other ORMs have considered this problem suggest it is a common design choice.
  4. Is the polymorphic solution (possibly via SQLAlchemy) an option? Is this being considered for Django 1.9+?

NB: I have read the docs for and tested https://github.com/chrisglass/django_polymorphic, but it does not appear to work for 'abstract base class' inheritance (i.e. it is only for multi-table). If I did choose to do multi-table inheritance then django-polymorphic would be good for the query side of my problem and I guess my model-linking problem would disappear.

NB: This is a similar question to this one, but provides more detail.

Community
  • 1
  • 1
Doddie
  • 1,393
  • 13
  • 21
  • Well, choose [your poison](http://stackoverflow.com/questions/3579079/how-can-you-represent-inheritance-in-a-database#answer-3579462). `abstract = True` avoids the base-table joins, but you also opt for not being able to have an in-db representation of the base model at all (meaning you can also not query the abstract models). `abstract = True` essentially means *don't realize this model, it's only a template*. – dhke Sep 16 '15 at 10:50
  • I guess that is my issue, I'm not sure which will be more poisonous. As far as 'less' poisonous options go, I am asking about if there is any accepted way to provide the Python-level abstraction over the concrete-table choice. Kind of like what is suggested here: [SLQAlchemy's declarative extention's union on concrete table inheritance](http://docs.sqlalchemy.org/en/rel_1_0/orm/extensions/declarative/inheritance.html#concrete-table-inheritance). What do Django devs choose when presented with this problem? – Doddie Sep 16 '15 at 11:09
  • 1
    I don't remember having ever read any "recommendation" against concrete inheritance in Django's doc ??? Now, while abstract inheritance + unions would probably be a fine thing it's not actually implemented in Django's ORM so your solutions are mainly using concrete inheritance + django-polymorphic or implementing abstract inheritance + unions in Django's ORM. – bruno desthuilliers Sep 16 '15 at 11:23
  • If push comes to shove, you can always instruct polymorphic [not](https://django-polymorphic.readthedocs.org/en/latest/advanced.html#non-polymorphic-queries) to do the subclass resolution. I have not tested it, but *union on concrete tables* should mean a big performance hit when you access common attributes (because you need to union over multiple tables instead of a simple join). – dhke Sep 16 '15 at 12:06
  • @brunodesthuilliers [This question](http://stackoverflow.com/questions/23466577/should-i-avoid-multi-table-concrete-inheritance-in-django-by-any-means) has some of the recommendations. – Doddie Sep 16 '15 at 12:14
  • @Doddie these are not part of Django's official documentation are they ? But anyway: yes of course concrete inheritance doesn't come for free, and that should be obvious to anyone having any experience in relational databases. It doesn't mean you shouldn't use it when that's the right solution for the problem at hand. – bruno desthuilliers Sep 16 '15 at 12:26
  • Can you take a look at this https://django-model-utils.readthedocs.org/en/latest/managers.html and see if it solves your problem? I've been using it and works well. – Shang Wang Sep 16 '15 at 13:10
  • @ShangWang Your link will help if I elect to go down the concrete inheritance path. – Doddie Sep 16 '15 at 14:24
  • I posted a similar question (http://stackoverflow.com/questions/32500356/how-to-implement-variant-records-or-object-subclasses) a while ago. No answers. However, I mentioned a couple of other possibilities: using a single wide table with a lot of mostly blank or null fields, and if using Postgres and Django 1.8+, a table with an hstore field. Don't have anything to add to what's there, yet. – nigel222 Sep 16 '15 at 19:40
  • @nigel222 Yes, your question is pretty much the same. I will certainly look into the hstore option (I don't like the wide-table choice). I guess we are both waiting for someone with war stories to help? – Doddie Sep 16 '15 at 20:06

2 Answers2

9

Ok, so I think I'm going to answer my own questions...

  1. Is this a good case for 'multi-table' inheritance?

    It appears so. Although there are a few places that recommend against 'multi-table' inheritance (listed here for example), some counterpoints are:

    • @Bruno Desthuilliers points out that these opinions are not from the 'official' documentation, and by extension, he implies that 'multi-table' is a perfectly good feature available for one to use.

    • My reading of @dhke's links and comments is that you have to choose one option, and that the 'multi-table' option is the only way databases truly support inheritance. I.e. Even with the polymorphic tricks of tools like Hibernate or SQLAlchemy, you are still choosing whether to JOIN tables ('multi-table' option) for object lookup or to UNION tables ('abstract base' / 'polymorphic' options) for set creation.

    • @dhke also points out that it is probably better to use the 'multi-table' option and tell a library like django-polymorphic not to do subclass resolution when looking up the 'whole set' rather than have the database do a UNION over all of the tables (as would be required for 'whole set' lookup with the 'abstract base class' option).

  2. Am I trying too hard to force the type inheritance? Should I just get rid of Tool? If yes, then do I have to create a *ToolGroup model for each subclass?

    No, it doesn't seem that way. The two uses of the Tool interface that I presented have different needs:

    • The ToolGroup / hierarchical-grouping use case is a good one for retaining the inherited Tool class. This would get very ugly if you had to create a type-specific set of classes for every type of tool

    • The ToolAttribute also makes a good case for the super class, except if you are able to use things like the HSTORE field type (provided by Postgres, I'm not sure about other backends). This link gives a good rundown, and it is probably what I will do here (Thanks to @nigel222 for the research that went into the question!).

  3. What is the current (Django 1.8) accepted way around this? Surely I am not the first person to build a relational system in Django ;-) And the fact that other ORMs have considered this problem suggest it is a common design choice.

    This is now an irrelevant question. Basically they don't worry about it.

  4. Is the polymorphic solution (possibly via SQLAlchemy) an option? Is this being considered for Django 1.9+?

    Not that I can tell.

Community
  • 1
  • 1
Doddie
  • 1,393
  • 13
  • 21
2

The case that led me to this question is a model like this: 

class PurchasableItem(models.Model):

    class Meta:
        abstract = True


class Cheesecake(PurchasableItem):
    pass


class Coffee(PurchasableItem):
    pass

The workaround I used is turning the parent class into an attribute:

class PurchasableItem(models.Model):

    class Meta:
        abstract = False


class Cheesecake(models.Model):
    purchasable_item = models.OneToOneField(PurchasableItem, on_delete=models.CASCADE)


class Coffee(models.Model):
    purchasable_item = models.OneToOneField(PurchasableItem, on_delete=models.CASCADE)

This way, I can get both the behavior and querying functionality.

Pavel Vergeev
  • 3,060
  • 1
  • 31
  • 39