String of namedtuples to list

Question

How would I convert string of namedtuples to a list?

The problem is I have to store a list of namedtuples in a column in SQLite, which (obviously) doesn't support the format. I thought of just converting it into a string. However, since my tuple is a namedtuple, I don't know how to go from the string to list again.

>>> Point = namedtuple("Point", "x y", verbose = False)
>>> p = Point(3, 5)
>>> points = []
>>> points.append(Point(4, 7))
>>> points.append(Point(8, 9))
>>> points.append(p)
>>> p.x
3
>>> print points
[Point(x=4, y=7), Point(x=8, y=9), Point(x=3, y=5)]

My list of named tuples is something like this^^^^, but it has 6 arguments instead of the 2 shown above. Edit - the arguments are booleans, ints, and strings.

I tried mapping, but i got the following error:

>>> string = str(points)
>>> l = string.strip("[]")
>>> p = map(Point._make, l.split(", "))

Traceback (most recent call last):
File "<pyshell#31>", line 1, in <module>
p = map(Point._make, l.split(", "))
File "<string>", line 17, in _make
TypeError: Expected 2 arguments, got 9

I'm open to other simpler ways to do this.

Answer 1

Ultimately, how to do this may be a matter of taste.

JSON

Json can be a good use because, unlike pickle, it is usable beyond python. Your object is serialized in a widely supported, easily repurposed format.

>>> import json  # simple json is better bit I didn't want to force an install
>>> from collections import namedtuple
>>> Point = namedtuple("Point", "x y", verbose = False)
>>> p = Point(3,4)
>>> json.dumps(p._asdict())
'{"x": 3, "y": 4}'
>>> s = json.dumps(p._asdict())
>>> json.loads(s)  # not there yet cause thisis a dict
{u'y': 4, u'x': 3}   # but it is a dict that can create a Point
>>> Point(**json.loads(s))
Point(x=3, y=4)    

Pickle

pickle will not work unless you define a attribute state (see __getstate__ in the docs). This is "Nicer" in the load phase, following from above:

import pickle

# Point.__getstate__=lambda self: self._asdict() # not needed as per @simon's comment thx simon
>>> pickle.dumps(p)
"ccopy_reg\n_reconstructor\np0\n(c__main__\nPoint\np1\nc__builtin__\ntuple\np2\n(I3\nI4\ntp3\ntp4\nRp5\nccollections\nOrderedDict\np6\n((lp7\n(lp8\nS'x'\np9\naI3\naa(lp10\nS'y'\np11\naI4\naatp12\nRp13\nb."
s = pickle.dumps(p)
>>> pickle.loads(s)
Point(x=3, y=4)

eval

I would discourage any use of eval or exec. If you do go down that route check out ast.literal_eval() and checkout some of the SO related answers like safety of python eval

Answer 2

Based on Phil Cooper answer, you can store your objects in json format:

>>> import json

>>> points_str = json.dumps([x._asdict() for x in points])
[{"x": 4, "y": 7}, {"x": 8, "y": 9}, {"x": 1, "y": 2}]

>>> points2 = [Point(**x) for x in json.loads(points_str)]
[Point(x=4, y=7), Point(x=8, y=9), Point(x=1, y=2)]

another strange way to do it is to use exec:

>>> points_str = repr(points)
'[Point(x=4, y=7), Point(x=8, y=9), Point(x=1, y=2)]'

>>> exec "points2 = %s" % points
>>> points2
[Point(x=4, y=7), Point(x=8, y=9), Point(x=1, y=2)]

Answer 3

I'd recommend you to use modules like pickle that allow to to store python objects in files.

By the way I am not sure if namedtuple will work with pickle, if that's the case and source of the data is not unknown then you can also use eval with repr:

help on repr:

>>> print repr.__doc__
repr(object) -> string

Return the canonical string representation of the object.
For most object types, eval(repr(object)) == object.

Example:

>>> repr(points)
'[Point(x=4, y=7), Point(x=8, y=9), Point(x=3, y=5)]'
>>> eval(repr(points))
[Point(x=4, y=7), Point(x=8, y=9), Point(x=3, y=5)]