How to convert a string containing a list of dict into Python object?

Question

ugly = "[{'ride': 1, 'pickup_time': datetime(2016, 3, 17, 15, 36, 35, 976202)},
         {'ride': 2, 'pickup_time': datetime.datetime(2016, 3, 17, 15, 41, 35, 976202)}]"
# The actual variable contains a lot more dictionnaries...`

I'd like to convert the ugly variable as a real Python object. I tried json.loads() and ast.literal_eval() but it's for one dict only.

That's why, before that, I tried to split this string into several dictionnaries, but the split() method has only one delimiter, so it seems I could need a REGEX to do this.

What is the simplest way to do it? Thanks.

How did your list of dictionaries become a string in the first place? — jDo, Mar 17 '16 at 18:35
Just to note, `ast.literal_eval` would work fine with a list of dicts; it doesn't like the `datetime` call. — Aaron Christiansen, Mar 17 '16 at 18:36
@jDo Someone who prefers to store object as string rather than structured way in my company because we don't have time... — David Dahan, Mar 17 '16 at 18:42

score 11 · Accepted Answer · answered Mar 17 '16 at 18:45

Well first you should be consistent in you calls to datetime method. In first dict, you use datetime and in second datetime.datetime.

Whatever way you try to have Python eval the string, you cannot have at the same time datetime be a function (first dict) and a module (second). Once you fix that, you will be forced to use the evil eval function, because neither json.loads not ast.litteral_eval will accept a function. They are normally used precisely to avoid the evaluation to call any function...

But if this is what you want to do (and you are sure that ugly contains no harmful code), this works:

>>> ugly = "[{'ride': 1, 'pickup_time': datetime.datetime(2016, 3, 17, 15, 36, 35, 976202)},{'ride': 2, 'pickup_time': datetime.datetime(2016, 3, 17, 15, 41, 35, 976202)}]"
>>> import datetime
>>> dlist = eval(ugly)
>>> dlist
[{'ride': 1, 'pickup_time': datetime.datetime(2016, 3, 17, 15, 36, 35, 976202)}, {'ride': 2, 'pickup_time': datetime.datetime(2016, 3, 17, 15, 41, 35, 976202)}]

Thanks! (and the other ones too for their answer) – David Dahan Mar 18 '16 at 10:07 — David Dahan, Mar 18 '16 at 10:07

score 6 · Answer 2 · answered Mar 17 '16 at 18:35

6

You can use eval

a = eval(ugly)

At this point a is a list of dictionaries and I am sure you got it from there.

answered Mar 17 '16 at 18:35

kpie

9,588
5
28
50

TypeError: 'module' object is not callable – David Dahan Mar 17 '16 at 18:40
2

Please note that `eval` is dangerous. Use caution while you are using it. See http://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html – Bhargav Rao Mar 17 '16 at 18:41
1

@DavidD.you've got a `datetime` in one record and `datetime.datetime` in another. You need to make them consistent. – Mar 17 '16 at 18:42
Bummer, looks like an issue with not having datetime.datetime on pickup-time... like Martin so quickly pointed out. – kpie Mar 17 '16 at 18:42
`eval(ugly.replace(' datetime(', ' datetime.datetime('))` – Steven Rumbalski Mar 17 '16 at 18:54

score 0 · Answer 3 · answered Mar 17 '16 at 18:50

You can use eval to get the array of dictionaries (after fixing the datetime issue), and then use setattr to turn the dictionaries into real class objects:

from datetime import datetime

ugly = "[{'ride': 1, 'pickup_time': datetime(2016, 3, 17, 15, 36, 35, 976202)}, {'ride': 2, 'pickup_time': datetime(2016, 3, 17, 15, 41, 35, 976202)}]"
array = eval(ugly)

class Ride(object):
    pass

rides = []
for record in array:
    ride = Ride()
    for k, v in record.iteritems():
        setattr(ride, k, v)
    rides.append(ride)

for ride in rides:
    print "ride: {0}: pickup_time: {1}".format(ride.ride, ride.pickup_time)

jDo · Answer 4 · 2016-03-17T20:21:22.280

Depending on how you want to import datetime:

import datetime

ugly = "[{'ride': 1, 'pickup_time': datetime(2016, 3, 17, 15, 36, 35, 976202)},{'ride': 2, 'pickup_time': datetime.datetime(2016, 3, 17, 15, 41, 35, 976202)}]"
ugly = ugly.replace(" datetime(", " datetime.datetime(")
ugly = eval(ugly)

or

from datetime import datetime

ugly = "[{'ride': 1, 'pickup_time': datetime(2016, 3, 17, 15, 36, 35, 976202)},{'ride': 2, 'pickup_time': datetime.datetime(2016, 3, 17, 15, 41, 35, 976202)}]"
ugly = ugly.replace("datetime.", "")
ugly = eval(ugly)

Both work without prior cleaning of ugly

score 0 · Answer 5 · answered Mar 17 '16 at 20:13

You could do it with a regex by extracting the datetimes and their keys only calling eval on those pairings:

from datetime import datetime
from ast import literal_eval
import re


def parse(ug):
    ug = ug.replace("datetime.", "")
    pairs = ("{{{}}}".format(p) for p in re.findall("('\w+':\s+datetime\(.*\))", ug))
    _parsed = literal_eval(re.sub("datetime\(.*\)","''", ug))
    for d in _parsed:
        d.update(eval(next(pairs)))
    return _parsed

The lists are going to be ordered so the correct pairing will be put back in the correct dicts:

In [4]: parse(ugly)
Out[4]: 
[{'pickup_time': datetime.datetime(2016, 3, 17, 15, 36, 35, 976202),
  'ride': 1},
 {'pickup_time': datetime.datetime(2016, 3, 17, 15, 41, 35, 976202),
  'ride': 2}]

How to convert a string containing a list of dict into Python object?

5 Answers5

Linked