Weird Issue when using dataclass and property together

Question

I ran into a strange issue while trying to use a dataclass together with a property.

I have it down to a minumum to reproduce it:

import dataclasses

@dataclasses.dataclass
class FileObject:
    _uploaded_by: str = dataclasses.field(default=None, init=False)
    uploaded_by: str = None

    def save(self):
        print(self.uploaded_by)

    @property
    def uploaded_by(self):
        return self._uploaded_by

    @uploaded_by.setter
    def uploaded_by(self, uploaded_by):
        print('Setter Called with Value ', uploaded_by)
        self._uploaded_by = uploaded_by

p = FileObject()
p.save()

This outputs:

Setter Called with Value  <property object at 0x7faeb00150b0>
<property object at 0x7faeb00150b0>

I would expect to get None instead of

Am I doing something wrong here or have I stumbled across a bug?

After reading @juanpa.arrivillaga answer I thought that making uploaded_by and InitVar might fix the issue, but it still return a property object. I think it is because of the this that he said:

the datalcass machinery interprets any assignment to a type-annotated variable in the class body as the default value to the created __init__.

The only option I can find that works with the default value is to remove the uploadedby from the dataclass defintion and write an actual __init__. That has an unfortunate side effect of requiring you to write an __init__ for the dataclass manually which negates some of the value of using a dataclass. Here is what I did:

import dataclasses

@dataclasses.dataclass
class FileObject:
    _uploaded_by: str = dataclasses.field(default=None, init=False)
    uploaded_by: dataclasses.InitVar=None
    other_attrs: str = None

    def __init__(self, uploaded_by=None, other_attrs=None):
        self._uploaded_by = uploaded_by
        self.other_attrs = other_attrs

    def save(self):
        print("Uploaded by: ", self.uploaded_by)
        print("Other Attrs: ", self.other_attrs)

    @property
    def uploaded_by(self):
        if not self._uploaded_by:
            print("Doing expensive logic that should not be repeated")
        return self._uploaded_by

p = FileObject(other_attrs="More Data")
p.save()

p2 = FileObject(uploaded_by='Already Computed', other_attrs="More Data")
p2.save()

Which outputs:

Doing expensive logic that should not be repeated
Uploaded by:  None
Other Attrs:  More Data
Uploaded by:  Already Computed
Other Attrs:  More Data

The negatives of doing this:

• You have to write boilerplate __init__ (My actual use case has about 20 attrs)
• You lose the uploaded_by in the __repr__, but it is there in _uploaded_by
• Calls to asdict, astuple, dataclasses.replace aren't handled correctly

So it's really not a fix for the issue

I have filed a bug on the Python Bug Tracker: https://bugs.python.org/issue39247

Answer 1

So, unfortunately, the @property syntax is always interpreted as an assignment to uploaded_by (since, well, it is). The dataclass machinery is interpreting that as a default value, hence why it is passing the property object! It is equivalent to this:

In [11]: import dataclasses
    ...:
    ...: @dataclasses.dataclass
    ...: class FileObject:
    ...:     uploaded_by: str
    ...:     _uploaded_by: str = dataclasses.field(repr=False, init=False)
    ...:     def save(self):
    ...:         print(self.uploaded_by)
    ...:
    ...:     def _get_uploaded_by(self):
    ...:         return self._uploaded_by
    ...:
    ...:     def _set_uploaded_by(self, uploaded_by):
    ...:         print('Setter Called with Value ', uploaded_by)
    ...:         self._uploaded_by = uploaded_by
    ...:     uploaded_by = property(_get_uploaded_by, _set_uploaded_by)
    ...: p = FileObject()
    ...: p.save()
Setter Called with Value  <property object at 0x10761e7d0>
<property object at 0x10761e7d0>

Which is essentially acting like this:

In [13]: @dataclasses.dataclass
    ...: class Foo:
    ...:     bar:int = 1
    ...:     bar = 2
    ...:

In [14]: Foo()
Out[14]: Foo(bar=2)

I don't think there is a clean way around this, and perhaps it could be considered a bug, but really, not sure what the solution should be, because essentially, the datalcass machinery interprets any assignment to a type-annotated variable in the class body as the default value to the created __init__. You could perhaps either special-case the @property syntax, or maybe just the property object itself, so at least the behavior for @property and x = property(set_x, get_x) would be consistent...

To be clear, the following sort of works:

In [22]: import dataclasses
    ...:
    ...: @dataclasses.dataclass
    ...: class FileObject:
    ...:     uploaded_by: str
    ...:     _uploaded_by: str = dataclasses.field(repr=False, init=False)
    ...:     @property
    ...:     def uploaded_by(self):
    ...:         return self._uploaded_by
    ...:     @uploaded_by.setter
    ...:     def uploaded_by(self, uploaded_by):
    ...:         print('Setter Called with Value ', uploaded_by)
    ...:         self._uploaded_by = uploaded_by
    ...:
    ...: p = FileObject(None)
    ...: print(p.uploaded_by)
Setter Called with Value  None
None

In [23]: FileObject()
Setter Called with Value  <property object at 0x1086debf0>
Out[23]: FileObject(uploaded_by=<property object at 0x1086debf0>)

But notice, you cannot set a useful default value! It will always take the property... Even worse, IMO, if you don't want a default value it will always create one!

EDIT: Found a potential workaround!

This should have been obvious, but you can just set the property object on the class.

import dataclasses
import typing
@dataclasses.dataclass
class FileObject:
    uploaded_by:typing.Optional[str]=None

    def _uploaded_by_getter(self):
        return self._uploaded_by

    def _uploaded_by_setter(self, uploaded_by):
        print('Setter Called with Value ', uploaded_by)
        self._uploaded_by = uploaded_by

FileObject.uploaded_by = property(
    FileObject._uploaded_by_getter,
    FileObject._uploaded_by_setter
)
p = FileObject()
print(p)
print(p.uploaded_by)

Answer 2

The alternative take on @juanpa.arrivillaga solution of setting properties, which may look a tad more object-oriented, initially proposed at python-list by Peter Otten

import dataclasses
from typing import Optional


@dataclasses.dataclass
class FileObject:
    uploaded_by: Optional[str] = None

class FileObjectExpensive(FileObject):
    @property
    def uploaded_by(self):
        return self._uploaded_by

    @uploaded_by.setter
    def uploaded_by(self, uploaded_by):
        print('Setter Called with Value ', uploaded_by)
        self._uploaded_by = uploaded_by

    def save(self):
        print(self.uploaded_by)

p = FileObjectExpensive()
p.save()
p2 = FileObjectExpensive(uploaded_by='Already Computed')
p2.save()

This outputs:

Setter Called with Value  None
None
Setter Called with Value  Already Computed
Already Computed

To me this approach, while not being perfect in terms of removing boilerplate, has a little more readability and explicitness in the separation of the pure data container and behaviour on that data. And it keeps all variables' and properties' names the same, so readability seems to be the same.

Answer 3

Slightly modified solution from original question using metaclass approach - hope it helps :)

from __future__ import annotations
import dataclasses
from dataclass_wizard import property_wizard

@dataclasses.dataclass
class FileObject(metaclass=property_wizard):
    uploaded_by: str | None
    # uncomment and use for better IDE support
    # _uploaded_by: str | None = dataclasses.field(default=None)

    def save(self):
        print(self.uploaded_by)

    @property
    def uploaded_by(self):
        return self._uploaded_by

    @uploaded_by.setter
    def uploaded_by(self, uploaded_by):
        print('Setter Called with Value ', uploaded_by)
        self._uploaded_by = uploaded_by

p = FileObject()
p.save()

This outputs (as I assume is desired behavior):

Setter Called with Value  None
None

Edit (4/1/22): Adding clarification for future viewers. The dataclass-wizard is a library I've created to tackle the issue of field properties with default values in dataclasses, among other things. It can be installed with pip:

$ pip install dataclass-wizard

---

If you are interested in an optimized approach that relies only on stdlib, I created a simple gist which uses a metaclass approach.

Here's general usage below. This will raise an error as expected when the name field is not passed in to constructor:

@dataclass
class Test(metaclass=field_property_support):
    my_int: int
    name: str
    my_bool: bool = True

    @property
    def name(self):
        return self._name

    @name.setter
    def name(self, val):
        print(f'Setting name to: {val!r}')
        self._name = val

Answer 4

For completeness, and with credit to @juanpa.arrivillaga, here is a proposed answer to the original question which uses decorators.

It works at least with the use cases shown, and I prefer it to the method described here because it lets us assign a default value using the normal dataclass idiom.

The key is to defeat the @dataclass machinery by creating the getter and setter on a 'dummy' property (here '_uploaded_by') and then overwriting the original attribute from outside the class.

Maybe someone more knowledgeable than I can find a way to do the overwrite within __post_init__() ...

import dataclasses


@dataclasses.dataclass
class FileObject:
    uploaded_by: str = None

    def save(self):
        print(self.uploaded_by)

    @property
    def _uploaded_by(self):
        return self._uploaded_by_attr

    @_uploaded_by.setter
    def _uploaded_by(self, uploaded_by):
        # print('Setter Called with Value ', uploaded_by)
        self._uploaded_by_attr = uploaded_by


# --- has to be called at module level ---
FileObject.uploaded_by = FileObject._uploaded_by


def main():
    p = FileObject()
    p.save()                            # displays 'None'

    p = FileObject()
    p.uploaded_by = 'foo'
    p.save()                            # displays 'foo'

    p = FileObject(uploaded_by='bar')
    p.save()                            # displays 'bar'


if __name__ == '__main__':
    main()

Answer 5

Based on the solution of @juanpa.arrivillaga, I wrote the following function that makes it reusable as additional decorator:

from dataclasses import fields

def dataprops(cls):
    """A decorator to make dataclasses fields acting as properties
    getter and setter methods names must initate with `get_` and `set_`"""
    
    for field in fields(cls):
        setattr(cls,
                field.name,
                property(
                    getattr(cls,f'get_{field.name}'),
                    getattr(cls,f'set_{field.name}')
                    )
                )
    return cls

Simple usage:

from dataclasses import dataclass

@dataprops
@dataclass
class FileObject:
    uploaded_by: str = "no_one"

    def save(self):
        print(self.uploaded_by)

    def get_uploaded_by(self):
        return self._uploaded_by

    def set_uploaded_by(self, uploaded_by):
        print('Setter Called with Value: ', uploaded_by)
        self._uploaded_by = uploaded_by

Output results:

p = FileObject()
p.save()

# output:
# Setter Called with Value:  no_one
# no_one

p = FileObject("myself")
p.save()

# output:
# Setter Called with Value:  myself
# myself