Python breaks parsing json with characters \"

Question

I'm trying to parse json string with an escape character (Of some sort I guess)

{
    "publisher": "\"O'Reilly Media, Inc.\""
}

Parser parses well if I remove the character \" from the string,

the exceptions raised by different parsers are,

json

  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting , delimiter: line 17 column 20 (char 392)

ujson

ValueError: Unexpected character in found when decoding object value

How do I make the parser to escape this characters ?

update: ps. json is imported as ujson in this example

This is what my ide shows

comma is just added accidently, it has no trailing comma at the end of json, json is valid

the string definition.

How did you define the input string? When you *print* the string, do you still see the backslashes? I cannot reproduce this at all. — Martijn Pieters, Oct 01 '15 at 17:39
If the input string is within Python source, you may need to escape metacharacters there first. Consider raw strings if that's the case. — Brian Cain, Oct 01 '15 at 17:41
@MartijnPieters the comma is added accidentally, the issue is the characters `/"` — All Іѕ Vаиітy, Oct 01 '15 at 17:51
can you show us the method where you are initiating the parse? — Shawn Mehan, Oct 01 '15 at 17:52
@Marty: note that your screenshot shows you are defining a *Python string*, so the `\"` is *interpreted as a Python string literal escape*. The backlashes are gone in the actual value you produced. — Martijn Pieters, Oct 01 '15 at 17:59

Martijn Pieters · Accepted Answer · 2015-10-01T17:57:59.143

You almost certainly did not define properly escaped backslashes. If you define the string properly the JSON parses just fine:

>>> import json
>>> json_str = r'''
... {
...     "publisher": "\"O'Reilly Media, Inc.\""
... }
... '''  # raw string to prevent the \" from being interpreted by Python
>>> json.loads(json_str)
{u'publisher': u'"O\'Reilly Media, Inc."'}

Note that I used a raw string literal to define the string in Python; if I did not, the \" would be interpreted by Python and a regular " would be inserted. You'd have to double the backslash otherwise:

>>> print '\"'
"
>>> print '\\"'
\"
>>> print r'\"'
\"

Reencoding the parsed Python structure back to JSON shows the backslashes re-appearing, with the repr() output for the string using the same double backslash:

>>> json.dumps(json.loads(json_str))
'{"publisher": "\\"O\'Reilly Media, Inc.\\""}'
>>> print json.dumps(json.loads(json_str))
{"publisher": "\"O'Reilly Media, Inc.\""}

If you did not escape the \ escape you'll end up with unescaped quotes:

>>> json_str_improper = '''
... {
...     "publisher": "\"O'Reilly Media, Inc.\""
... }
... '''
>>> print json_str_improper

{
    "publisher": ""O'Reilly Media, Inc.""
}

>>> json.loads(json_str_improper)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting , delimiter: line 3 column 20 (char 22)

Note that the \" sequences now are printed as ", the backslash is gone!

have updated the question, sorry for the accidental trailing comma, The json is valid. — All Іѕ Vаиітy, Oct 01 '15 at 17:49
thank you, defining the string as raw (`r`) as per this answer solved the problem. note the question's error message was messed up, updated. — All Іѕ Vаиітy, Oct 01 '15 at 18:10

Shawn Mehan · Answer 2 · 2015-10-01T18:03:18.280

Your JSON is invalid. If you have questions about your JSON objects, you can always validate them with JSONlint. In your case you have an object

{
"publisher": "\"O'Reilly Media, Inc.\"",
}

and you have an extra comma indicating that something else should be coming. So JSONlint yields

Parse error on line 2: ...edia, Inc.\"", } ---------------------^ Expecting 'STRING'

which would begin to help you find where the error was.

Removing the comma for

{
"publisher": "\"O'Reilly Media, Inc.\""
}

yields

Valid JSON

Update: I'm keeping the stuff in about JSONlint as it may be helpful to others in the future. As for your well formed JSON object, I have

import json

d = {
    "publisher": "\"O'Reilly Media, Inc.\""
    }

print "Here is your string parsed."
print(json.dumps(d))

yielding

Here is your string parsed. {"publisher": "\"O'Reilly Media, Inc.\""}

Process finished with exit code 0

sorry for the accidental trailing comma, the json is valid, have posted a screenshot. — All Іѕ Vаиітy, Oct 01 '15 at 17:50

Python breaks parsing json with characters \"

2 Answers2