Python: Extending a predefined named tuple

Question

I have the following named tuple:

from collections import namedtuple
ReadElement = namedtuple('ReadElement', 'address value')

and then I want the following:

LookupElement = namedtuple('LookupElement', 'address value lookups')

There is duplication between the two namedtuples, how can I subclass ReadElement to contain an additional field?

class LookupElement(ReadElement):
    def __new__(self, address, value, lookups):
        self = super(LookupElement, self).__new__(address, value)
        l = list(self)
        l.append(lookups)
        return tuple(l)

However the tuple is created there an then in the new statement, if I modify self to be a list I will loose type information, how can I avoid this?

Martijn Pieters · Accepted Answer · 2018-05-17T11:59:22.043

41

You can subclass a namedtuple-produced class, but you need to study the generated class more closely. You'll need to add another __slots__ attribute with the extra fields, update the _fields attribute, create new __repr__ and _replace methods (they hardcode the field list and class name) and add extra property objects for the additional fields. See the example in the documentation.

That's all a little too much work. Rather than subclass, I'd just reuse the somenamedtuple._fields attribute of the source type:

LookupElement = namedtuple('LookupElement', ReadElement._fields + ('lookups',))

The field_names argument to the namedtuple() constructor doesn't have to be a string, it can also be a sequence of strings. Simply take the _fields and add more elements by concatenating a new tuple.

Demo:

>>> from collections import namedtuple
>>> ReadElement = namedtuple('ReadElement', 'address value')
>>> LookupElement = namedtuple('LookupElement', ReadElement._fields + ('lookups',))
>>> LookupElement._fields
('address', 'value', 'lookups')
>>> LookupElement('addr', 'val', 'lookup') 
LookupElement(address='addr', value='val', lookups='lookup')

This does mean that the extended type is not a subclass of the base type. If you must have a class hierarchy, then rather than try to make named tuples fit that model, I'd switch to using dataclasses instead. Dataclasses can serve the same purpose in most usecases named tuples are used for, but can easily be subclassed.

edited May 17 '18 at 11:59

answered Feb 13 '15 at 13:22

Martijn Pieters

1,048,767
296
4,058
3,343

This is what the docs suggest doing, but what if you've got a custom namedtuple with a generated field? – Ethereal Jan 03 '18 at 01:17
@Ethereal: all namedtuple classes are custom. The `_fields` attribute will still reflect the actual fields on the class. – Martijn Pieters Jan 03 '18 at 09:10
I'm finding that this is not true, _fields doesn't contain the generated field! See eg the `hypot` example in the docs. – Ethereal Jan 03 '18 at 23:37
1

@Ethereal so you are talking about a `property` object. I’d never call that a generated field. No, additional attributes on a subclass of a `namedtuple` object are not reflected in the `_fields` attribute. If you need to share additional methods or properties, consider putting those on a mix-in class instead and reuse these between multiple namedtuple classes. – Martijn Pieters Jan 04 '18 at 01:11
@MartijnPieters correct, sorry for the nomenclature confusion and the suggestion. – Ethereal Jan 05 '18 at 01:51

score 7 · Answer 2 · answered Jun 02 '20 at 19:47

Extending Martijn Pieters' answer: there is a way to make the new namedtuple class a subclass of the other, without having to hack. Simply create the new namedtuple separately, and then use its __new__ method instead of using super:

from collections import namedtuple

class ReadElement(namedtuple('ReadElement', ('address', 'value'))):
    def compute(self):
        return self.value + 1

_LookupElement = namedtuple('_LookupElement', ReadElement._fields + ('lookups',))

class LookupElement(_LookupElement, ReadElement):
    def __new__(self, address, value, lookups):
        return _LookupElement.__new__(LookupElement, address, value, lookups)

assert issubclass(LookupElement, ReadElement)
l = LookupElement('ad', 1, dict())
assert isinstance(l, ReadElement)
assert l.compute() == 2

It seems that this also works without even overriding __new__ !

from collections import namedtuple

class ReadElement(namedtuple('ReadElement', ('address', 'value'))):
    def compute(self):
        return self.value + 1

class LookupElement(namedtuple('LookupElement', ReadElement._fields + ('lookups',)),
                    ReadElement):
    """nothing special to do"""
    pass

score 4 · Answer 3 · answered Jun 11 '18 at 15:15

It's quite easy to knock something together that allows you to compose namedtuples from other namedtuples as well as introduce new fields.

def extended_namedtuple(name, source_fields):
    assert isinstance(source_fields, list)
    new_type_fields = []
    for f in source_fields:
        try:
            new_type_fields.extend(f._fields)
        except:
            new_type_fields.append(f) 
    return namedtuple(name, new_type_fields) 

# source types
Name = namedtuple('Name', ['first_name', 'last_name'])
Address = namedtuple('Address', ['address_line1', 'city'])
# new type uses source types and adds additional ID field
Customer = extended_namedtuple('Customer', ['ID', Name, Address])
# using the new type
cust1 = Customer(1, 'Banana', 'Man', '29 Acacia Road', 'Nuttytown')
print(cust1)

This outputs the following :

Customer(ID=1, first_name='Banana', last_name='Man', address_line1='29 Acacia Road', city='Nuttytown')

Python: Extending a predefined named tuple

3 Answers3

Linked