How to parse improperly escaped string in python

Asked Dec 12 '14 at 18:34

Active Dec 12 '14 at 19:14

Viewed 31 times

I'm currently working with a largish (2 gig) json file that I need to parse into Python.

The problem is that some of the data is improperly escaped, which is giving me an error.

Here is my code

import json

data = []
with open('full.json') as f:
for line in f:
    data.append(json.loads(line))

which is giving me the following error

ValueError: Invalid \escape: line 1 column 64 (char 63)

I printed out the file and found the line that is giving me trouble

{"page_viewed":"Truck Repair Form - \342\204\242"}

I know its the \342\204\242 section that is holding me back, I just need to figure out how to properly escape the backslashes.

edited Dec 12 '14 at 19:14

Martijn Pieters

asked Dec 12 '14 at 18:34

jasnonaz

Those are probably character escapes encoding *octal* values; `\342` is U+00E2 or `â` for example. – Martijn Pieters Dec 12 '14 at 18:37
Here it is with less detail taken out. "page_viewed":"Truck Repair Form - \342\204\242" – jasnonaz Dec 12 '14 at 18:52
Is that perhaps `Truck Repair Form - ™` in UTF-8? – Martijn Pieters Dec 12 '14 at 19:00
My solution in the other answer will work for you if you use `chr()` instead of `unichr()` in the `replace_with_codepoint` function. – Martijn Pieters Dec 12 '14 at 19:13

0 Answers0