1

I have a string representation of a dictionary. It's a nested dictionary, even more complex, of which the record_time key has a datetime value like below:

d = """{
    "motion_measure": {"INCAR": 69, "RANDOM": 63, "UNKNOWN": 62, "BIKING": 57, "WALKING": 48, "RUNNING": 41, "SEDENTARY": 0},
    "samples": [0, 1.1791444, 11.036073],
    "record_time": datetime.datetime(2018, 3, 26, 10, 3, 17, 441000)
    }"""

I tried two methods as suggested in this related question: 

import ast

ast.literal_eval(d)

ast.literal_eval doesn't work because it can't handle the datetime value.

import json

json.loads(d)

json.loads doesn't work either because datetime format is not deserializable.

Does anybody know a good way to convert it back into a dict or json using Python? I got this strange record format from OrientDB.

eric R
  • 321
  • 2
  • 7

2 Answers2

1

Since your string is nearly json, you could just clean it up a bit and use json.loads (and avoid eval).

For instance, if you are ok with converting your datetime values to lists, then one way would be to replace all the instances of datetime.datetime(...) in your string before converting to a dict (example below uses regex to handle the replacement). Then, you could use datetime to work with the datetime lists in the resulting dict.

For example (added an extra datetime to your input string to ensure that the replacement handles multiple datetime occurrences and datetimes without specified milliseconds, etc):

import datetime
import re
import json

s = """{
    "motion_measure": {"INCAR": 69, "RANDOM": 63, "UNKNOWN": 62, "BIKING": 57, "WALKING": 48, "RUNNING": 41, "SEDENTARY": 0},
    "samples": [0, 1.1791444, 11.036073],
    "record_time": datetime.datetime(2018, 3, 26, 10, 3, 17, 441000), 
    "another_time": datetime.datetime(2017, 3, 26, 10, 3)
    }"""

# re.sub to replace datetime; json.loads to convert to dict
d = json.loads(re.sub(r'datetime\.datetime\(([^)]*)\)', r'[\1]', s))

# datetime.datetime(...) to work with resulting datetime lists
date1 = datetime.datetime(*d['record_time'])
date2 = datetime.datetime(*d['another_time'])

print(d)
# OUTPUT (shown on multiple lines for readability)
# {
# 'motion_measure': {'INCAR': 69, 'RANDOM': 63, 'UNKNOWN': 62, 'BIKING': 57, 'WALKING': 48, 'RUNNING': 41, 'SEDENTARY': 0},
# 'samples': [0, 1.1791444, 11.036073],
# 'record_time': [2018, 3, 26, 10, 3, 17, 441000],
# 'another_time': [2017, 3, 26, 10, 3]
# }

print(date1)
print(date2)
# OUTPUT
# 2018-03-26 10:03:17.441000
# 2017-03-26 10:03:00
benvc
  • 14,448
  • 4
  • 33
  • 54
  • great! thanks for the answer! I also think of replacing all the datetime with a quote mark, so I can proceed forward using json.loads method. But I was stuck with the regex, could you explain a little bit on how the regex magic works? @benvc – eric R Oct 19 '18 at 13:47
  • @ericR - `re.sub` uses regex for string replacement. The `datetime\.datetime\(` bit matches "datetime.datetime(" (backslashes are escaping special characters in regex). The `([^)]*)` bit matches a "capture group" (getting the values we want for the string replacement) - the bracketed part is a negated character set followed by an asterisk that tells it to grab everything up to a close parenthesis. The last bit `\)` just matches the final close parenthesis. The `r'"\1"'` parameter tells re.sub to replace the string matched by the regex with the first (and only) capture group we matched. – benvc Oct 19 '18 at 14:03
  • @ericR - Also, try pasting your string and the`datetime\.datetime\(([^)]*)\)` regex into [regexr.com](https://regexr.com/) and you can read through the explanation it gives you for how the regex works (a nice tool for testing regex - there are several other similarly good ones you can try as well). – benvc Oct 19 '18 at 14:06
  • Then there is another problem. As the datetime resolution is not always the same(sometimes without %f or %s%f part), the format template used in the datetime.datetime.strptime doesn't match the record. Instead of checking the record format one by one, is there a existing solution to [Convert comma separated string to datetime](https://stackoverflow.com/questions/21548184/convert-comma-separated-string-to-datetime) – eric R Oct 19 '18 at 14:59
  • @ericR - see the edited answer to handle datetime values without seconds, milliseconds, etc. Important changes include converting the `datetime.datetime(...)` values in your input data to lists instead of strings (required minor change to `re.sub`) and just using `datetime.datetime()` to work with the datetime lists rather than using `strptime()` - better solution all around. – benvc Oct 19 '18 at 15:32
-1

If you are okay using 'eval' (it is dangerous, since it's executing data.), you can just do 'eval'.

import datetime
d = <... your string ...>
new_dict = eval(d)
Danyal
  • 528
  • 1
  • 7
  • 17