Did something about `namedtuple` change in 3.5.1?

Question

On Python 3.5.0:

>>> from collections import namedtuple
>>> cluster = namedtuple('Cluster', ['a', 'b'])
>>> c = cluster(a=4, b=9)
>>> c
Cluster(a=4, b=9)
>>> vars(c)
OrderedDict([('a', 4), ('b', 9)])

On Python 3.5.1:

>>> from collections import namedtuple
>>> cluster = namedtuple('Cluster', ['a', 'b'])
>>> c = cluster(a=4, b=9)
>>> c
Cluster(a=4, b=9)
>>> vars(c)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: vars() argument must have __dict__ attribute

Seems like something about namedtuple changed (or maybe it was something about vars()?).

Was this intentional? Are we not supposed to use this pattern for converting named tuples into dictionaries anymore?

@user2357112 - Yeah, I think this kind of change should be called out in the changelog. That's what made me think at first that it might've been a mistake. — Nick Chammas, Dec 08 '15 at 21:49
"Are we not supposed to use this pattern for converting named tuples into dictionaries anymore" I suppose we never *were* supposed to use this pattern, as `vars(x)` is documented to return `x.__dict__`, but I don't think it was ever documented that `namedtuple` instances have a `__dict__` attribute. Such a pattern *is* documented for the `Namespace` instances in the `argparse` module, I know, so perhaps that's where the tendency arises. In any case, I'm sure there are going to be some surprised developers whose code breaks, so this is a great question. — jme, Dec 08 '15 at 21:55
PS: perhaps the title of the post should be amended to include `vars`? This might make it easier for the influx of searches to find this post once people start upgrading to 3.5.1 en masse. — jme, Dec 08 '15 at 21:59
@jme - Yeah, looks like `._asdict()` is the correct way to do this. Regarding search, I think the mention of `vars()` in the question body and answers will be enough to help people find this question. — Nick Chammas, Dec 08 '15 at 22:02
Fair enough. Also, I stand corrected that the use of `vars()` was never documented: in the [Python 3.3 docs](https://docs.python.org/3.3/library/collections.html#collections.somenamedtuple._asdict) it states that `the same effect can be achieved by using the built-in vars() function` when referring to the functionality of `_asdict()`. — jme, Dec 08 '15 at 22:26
@NickChammas I don't think calling a method whose name begins with `_` from outside the class is ever the correct way to do something… — Blacklight Shining, Dec 08 '15 at 22:27
@BlacklightShining That is indeed a convention in python, but this is an exception to the rule. The [docs](https://docs.python.org/3/library/collections.html#collections.somenamedtuple._asdict) mentioned "to prevent conflicts with field names, the method and attribute names start with an underscore." — wim, Dec 08 '15 at 22:45
Unfortunately, Python doesn't seem to have an explicit policy forbidding backwards-incompatible changes in micro versions. Micro versions [are supposed to just be "bugfixes"](https://docs.python.org/devguide/devcycle.html), but without a policy forbidding it things like this happen repeatedly. — Jeremy, Dec 08 '15 at 23:29
@JeremyBanks arguably they *did* fix a bug. The bug allowed the first example to work, but breaks it in the second instance. Of course as jme mentioned, that bug was actually documented as a feature... — Wayne Werner, Dec 09 '15 at 01:49

ShadowRanger · Accepted Answer · 2015-12-08T21:52:20.170

Per Python bug #24931:

[__dict__] disappeared because it was fundamentally broken in Python 3, so it had to be removed. Providing __dict__ broke subclassing and produced odd behaviors.

Revision that made the change

Specifically, subclasses without __slots__ defined would behave weirdly:

>>> Cluster = namedtuple('Cluster', 'x y')
>>> class Cluster2(Cluster):
    pass
>>> vars(Cluster(1,2))
OrderedDict([('x', 1), ('y', 2)])
>>> vars(Cluster2(1,2))
{}

Use ._asdict().

Chad S. · Answer 2 · 2015-12-08T21:45:23.227

10

From the docs

Named tuple instances do not have per-instance dictionaries, so they are lightweight and require no more memory than regular tuples.

The docs (and help(namedtuple)) say to use c._asdict() to convert to a dict.

edited Dec 08 '15 at 21:45

answered Dec 08 '15 at 21:40

Chad S.

6,252
15
25

the interesting thing is that `vars` worked on python3.5.0 . . . – mgilson Dec 08 '15 at 21:43
The removal of the instance dict might have happened in 3.5.1 – Chad S. Dec 08 '15 at 21:44
Fair enough, that does seem to be the officially sanctioned way to do it. Shame is that much of the advice on [this question](http://stackoverflow.com/q/26180528/877069) recommending `vars()` is now incorrect. I wonder if this was intentional on the developers' part or just an accident. – Nick Chammas Dec 08 '15 at 21:45
`vars()` only works if the object has a `__dict__` attribute. If namedtuples had this dict they would require more memory than regular tuples. This was determined to be undesirable. Therefore the `__dict__` was removed, and thus `vars()` doesn't work. It is intended. – Chad S. Dec 08 '15 at 21:46
1

@ChadS. the `__dict__` attribute was already just a property, `namedtuple`s have `__slots__`. – jonrsharpe Dec 08 '15 at 21:47

score 8 · Answer 3 · answered Dec 08 '15 at 21:46

__dict__ was implemented as a @property and has been removed; you can see the change in the source code:

3.5.0:

def __repr__(self):
    'Return a nicely formatted representation string'
    return self.__class__.__name__ + '({repr_fmt})' % self

@property
def __dict__(self):
    'A new OrderedDict mapping field names to their values'
    return OrderedDict(zip(self._fields, self))

def _asdict(self):
    'Return a new OrderedDict which maps field names to their values.'
    return self.__dict__

def __getnewargs__(self):
    'Return self as a plain tuple.  Used by copy and pickle.'
    return tuple(self)

def __getstate__(self):
    'Exclude the OrderedDict from pickling'
    return None

3.5.1:

def __repr__(self):
    'Return a nicely formatted representation string'
    return self.__class__.__name__ + '({repr_fmt})' % self

def _asdict(self):
    'Return a new OrderedDict which maps field names to their values.'
    return OrderedDict(zip(self._fields, self))

def __getnewargs__(self):
    'Return self as a plain tuple.  Used by copy and pickle.'
    return tuple(self)

Did something about `namedtuple` change in 3.5.1?

3 Answers3

Linked