2

I need to store a complex number in a Django model. For those who forget, that simply means Z=R+jX where R and X are real numbers representing the real and imaginary components of the complex. There will be individual numbers, as well as lists that need to be stored. My searches so far haven't provided a good solution for lists, so I intend to let the database handle the list as individual records.

I see two options for storing a complex number:

1) create a custom field: class Complex(models.CharField) This would allow me to customize all aspects of the field, but that is a lot of extra work for validation if it is to be done properly. The major upside is that a single number is represented by a single field in the table.

2) let each complex number be represented by a row, with a float field for the real part, R, and another float field for the imaginary part, X. The downside to this approach is that I would need to write some converters that will create a complex number from the components, and vice versa. The upside is that the database will just see it as another record.

Surely this issue has been resolved in the past, but I can't find any good references, never mind one particular to Django.

This is my first crack at the field, it is based on another example I found that involved a few string manipulations. What isn't clear to me is how and where various validations should be performed (such as coercing a simple float into a complex number by adding +0j). I intend to add form functionality as well, so that the field behaves like a float field, but with additional restrictions or requirements.

I have not tested this code yet, so there may be issues with it. It is based on the code from an answer in this SO question. It appears after running the code that some changes took place in method names.

What is the most efficient way to store a list in the Django models?

class ComplexField(models.CharField):

    description = 'A complex number represented as a string'

    def __init__(self, *args, **kwargs):
        kwargs['verbose_name'] = 'Complex Number'
        kwargs['max_length'] = 64
        kwargs['default'] = '0+0j'

        super().__init__(*args, **kwargs)

    def to_python(self, value):
        if not value: return
        if isinstance(value, complex):
            return value
        return complex(value)

    def get_db_prep_value(self, value):
        if not value: return
        assert(isinstance(value, complex))
        return str(item)[1:-1]

    def value_to_string(self, obj):
        value = self._get_val_from_obj(obj)
        return self.get_db_prep_value(value)
Brian
  • 642
  • 7
  • 18
  • 1
    It sounds like you're describing exactly the scenario that a custom field is meant to address. It's better to have the to/from conversion as close to the database as possible so you don't have to remember to convert a string to a complex number in your templates or views. – Franey Mar 18 '19 at 20:17
  • Okay, I'll need to find a good example for custom fields. I'm surprised nothing has popped up yet in my searches. Most of what is returned for "complex" are things perceived as "difficult" so it doesn't make it easy. – Brian Mar 18 '19 at 21:04
  • So long as you don't need to query the ComplexField in any other way than as strings, your approach looks good. It gets hairier if you need to query in a more complex way. – AKX Mar 19 '19 at 14:43

3 Answers3

1

If your expression every time like R + jX you can make the following class

class ComplexNumber(models.Model):
    real_number = models.FloatField('Real number part')
    img_number = models.FloatFoeld('Img number part')

    def __str__(self):
        return complex(self.real_number, self.img_number)

and handle the outcome string with python see here

If you have multiple real and img part you can handle this with foreign keys or ManyToMany Fields. This maybe depend on your need.

Tobit
  • 406
  • 7
  • 19
  • Thanks for expanding on that @Tobit. Each record would be a number, so the only ForeignKey that should be required is the one tying it to the owner object. De-duplication seems like an idea as well, but I'm not going to touch that for a while. – Brian Mar 19 '19 at 12:50
  • 1
    @Brian, glad to help you. Maybe you can show us the finale solution or mark my answer as correct. – Tobit Mar 19 '19 at 13:19
  • This would incur an extra lookup for every ComplexNumber, which sounds kinda unfortunate. – AKX Mar 19 '19 at 13:55
  • @AKX can you explain what you mean? If you try to store something in a database you have to get it from the database? The class stores the full complex number. Otherwise you can use filter, annotations or something else to have only one lookup for multiple ComplexNumbers. I do not understand your comment. – Tobit Mar 19 '19 at 14:02
  • I assume the OP's intention is to be able to store complex numbers in a model the same way they would store, say, floats or decimals, i.e. in columns, not as foreign key references to another table. – AKX Mar 19 '19 at 14:05
  • with prefetch related you can avoid additional database queries in your template or your script https://docs.djangoproject.com/en/2.1/topics/db/optimization/#use-queryset-select-related-and-prefetch-related – Tobit Mar 19 '19 at 14:08
  • I'm aware of prefetch_related/select_related, yes, but even so, Django needing to retrieve those rows and construct the model objects will be kind of heavy. – AKX Mar 19 '19 at 14:09
  • Last comment from me, we have no struggle with prefetch related and a massive database. In my opinion the OP should decided if this is a valid way to handle his problem. If you have another option feel free to explain. – Tobit Mar 19 '19 at 14:16
  • Some is a little over my head but I'll try. I wouldn't be filtering or ordering by value, I only care about getting and setting the value according to the PK. The values would be used in other models as either individual numbers, or as elements of a list or tuple. Many (but not all) operations and validations apply from float, and it might be useful to define the add operation (for example) as `R_sum = R1+R2` and `X_sum=X1+X2` or something like that. I'll be honest that I haven't considered DB performance at all...getting data in and out of the DB is my primary concern at the moment. – Brian Mar 19 '19 at 15:17
1

Regarding custom fields, you've probably found the relevant part in the Django documentation already.

Whether a custom field (or a custom database type, see below) is worth the trouble really depends on what you need to do with the stored numbers. For storage and some occasional pushing around, you can go with the easiest sane solution (your number two as enhanced by Tobit).

With PostgreSQL, you have to possibility to implement custom types directly in the database, including operators. Here's the relevant part in the Postgres docs, complete with a complex numbers example, no less.

Of course you then need to expose the new type and the operators to Django. Quite a bit of work, but then you could do arithmetics with individual fields right in the database using Django ORM.

Endre Both
  • 5,540
  • 1
  • 26
  • 31
  • I'm using MariaDB, and would like to keep things DB agnostic for the moment. I'm looking for a solution that balances up front effort, with being able to get far enough in the project to redesign some things if needed (I'm still learning about Django, I feel like I've only scratched the surface). I'm leaning towards what @Tobit expanded on. I need to understand model methods a little more. – Brian Mar 19 '19 at 12:46
  • I ended up implementing a custom field, and it works well enough that I forgot about it. – Brian Apr 17 '19 at 13:40
  • That's great to hear! You could post your solution as an answer; it would be helpful to others tackling the problem. – Endre Both Apr 17 '19 at 13:44
1

To be honest, I'd just split the complex number into two float/decimal fields and add a property for reading and writing as a single complex number.

I came up with this custom field that ends up as a split field on the actual model and injects the aforementioned property too.

  • contribute_to_class is called deep in the Django model machinery for all the fields that are declared on the model. Generally, they might just add the field itself to the model, and maybe additional methods like get_latest_by_..., but here we're hijacking that mechanism to instead add two fields we construct within, and not the actual "self" field itself at all, as it does not need to exist as a database column. (This might break something, who knows...) Some of this mechanism is explained here in the Django wiki.

  • The ComplexProperty class is a property descriptor, which allows customization of what happens when the property it's "attached as" into an instance is accessed (read or written). (How descriptors work is a little bit beyond the scope of this answer, but there's a how-to guide in the Python docs.)

NB: I did not test this beyond running migrations, so things may be broken in unexpected ways, but at least the theory is sound. :)

from django.db import models


class ComplexField(models.Field):
    def __init__(self, **kwargs):
        self.field_class = kwargs.pop('field_class', models.FloatField)
        self.field_kwargs = kwargs.pop('field_kwargs', {})
        super().__init__(**kwargs)

    def contribute_to_class(self, cls, name, private_only=False):
        for field in (
            self.field_class(name=name + '_real', **self.field_kwargs),
            self.field_class(name=name + '_imag', **self.field_kwargs),
        ):
            field.contribute_to_class(cls, field.name)

        setattr(cls, name, ComplexProperty(name))


class ComplexProperty:
    def __init__(self, name):
        self.name = name

    def __get__(self, instance, owner):
        if not instance:
            return self
        real = getattr(instance, self.name + '_real')
        imag = getattr(instance, self.name + '_imag')
        return complex(real, imag)

    def __set__(self, instance, value: complex):
        setattr(instance, self.name + '_real', value.real)
        setattr(instance, self.name + '_imag', value.imag)


class Test(models.Model):
    num1 = ComplexField()
    num2 = ComplexField()
    num3 = ComplexField()


The migration for this looks like

migrations.CreateModel(
    name="Test",
    fields=[
        (
            "id",
            models.AutoField(
                auto_created=True, primary_key=True, serialize=False, verbose_name="ID"
            ),
        ),
        ("num1_real", models.FloatField()),
        ("num1_imag", models.FloatField()),
        ("num2_real", models.FloatField()),
        ("num2_imag", models.FloatField()),
        ("num3_real", models.FloatField()),
        ("num3_imag", models.FloatField()),
    ],
)

so as you can see, the three ComplexFields are broken down into six FloatFields.

AKX
  • 152,115
  • 15
  • 115
  • 172
  • Can you provide a little more verbose description? That sounds like my option 2) but I'm not at all familiar with properties, attributes in the sense of reading/writing them (in particular the `contribute_to_class` method). I understand for the property that you're just defining the setter and getter. Also, does this method of setter/getter extend well for form/input validation? – Brian Mar 19 '19 at 14:24
  • 1
    @Brian There – hope that helps. – AKX Mar 19 '19 at 14:30
  • I want to learn, so I appreciate the extra detail. I see the migration produces the `_real` and `_imag` columns, and the admin interface produced the two form fields. Is this due to the `contribute_to_class` method, or the property, or both? – Brian Mar 19 '19 at 14:37
  • The `contribute_to_class` method adds the two fields. To be honest, I didn't even really think about admin and other autogenerated UIs (such as ModelForms) ending up with the two separate fields, so this solution is definitely a little clunky in that sense. – AKX Mar 19 '19 at 14:38
  • To be honest, there will be few instances where the data will be in a form for display or editing. The view or model itself will be reading and writing to the DB as part of saving settings or results to the DB. Entering data as real and imaginary fields also avoids much other validation for spaces, valid digits, etc. which is a significant improvement over having to validate an entire string in a CharField or something like that. – Brian Mar 19 '19 at 15:20
  • How would I modify various properties of the individually rendered/managed field, like adding a default value to each `FloatField` for example? The individual fields work really well since they natively handle scientific notation without any additional code. – Brian Mar 19 '19 at 15:33
  • That's already supported by way of the `field_class`/`field_kwargs` kwargs to the `ComplexField`: `x = ComplexField(field_kwargs={'default': 8})` would make 8 the default for both `x_real` and `x_imag`. It would be a great idea to add processing to destructure `ComplexField(default=6+1j)` into components though :) – AKX Mar 20 '19 at 06:28
  • Thanks for responding. I will look into what you described. What you provided so far works really well. I'm not sure why someone down voted you. – Brian Mar 20 '19 at 13:40