jQuery.parseJSON throws “Invalid JSON” error due to escaped single quote in JSON

Question

I’m making requests to my server using jQuery.post() and my server is returning JSON objects (like { "var": "value", ... }). However, if any of the values contains a single quote (properly escaped like \'), jQuery fails to parse an otherwise valid JSON string. Here’s an example of what I mean (done in Chrome’s console):

data = "{ \"status\": \"success\", \"newHtml\": \"Hello \\\'x\" }";
eval("x = " + data); // { newHtml: "Hello 'x", status: "success" }

$.parseJSON(data); // Invalid JSON: { "status": "success", "newHtml": "Hello \'x" }

Is this normal? Is there no way to properly pass a single quote via JSON?

Answer 1

According to the state machine diagram on the JSON website, only escaped double-quote characters are allowed, not single-quotes. Single quote characters do not need to be escaped:

Update - More information for those that are interested:

---

Douglas Crockford does not specifically say why the JSON specification does not allow escaped single quotes within strings. However, during his discussion of JSON in Appendix E of JavaScript: The Good Parts, he writes:

JSON's design goals were to be minimal, portable, textual, and a subset of JavaScript. The less we need to agree on in order to interoperate, the more easily we can interoperate.

So perhaps he decided to only allow strings to be defined using double-quotes since this is one less rule that all JSON implementations must agree on. As a result, it is impossible for a single quote character within a string to accidentally terminate the string, because by definition a string can only be terminated by a double-quote character. Hence there is no need to allow escaping of a single quote character in the formal specification.

---

Digging a little bit deeper, Crockford's org.json implementation of JSON for Java is more permissible and does allow single quote characters:

The texts produced by the toString methods strictly conform to the JSON syntax rules. The constructors are more forgiving in the texts they will accept:

...

• Strings may be quoted with ' (single quote).

This is confirmed by the JSONTokener source code. The nextString method accepts escaped single quote characters and treats them just like double-quote characters:

public String nextString(char quote) throws JSONException {
    char c;
    StringBuffer sb = new StringBuffer();
    for (;;) {
        c = next();
        switch (c) {

        ...

        case '\\':
            c = this.next();
            switch (c) {

            ...

            case '"':
            case '\'':
            case '\\':
            case '/':
                sb.append(c);
                break;
        ...

At the top of the method is an informative comment:

The formal JSON format does not allow strings in single quotes, but an implementation is allowed to accept them.

So some implementations will accept single quotes - but you should not rely on this. Many popular implementations are quite restrictive in this regard and will reject JSON that contains single quoted strings and/or escaped single quotes.

---

Finally to tie this back to the original question, jQuery.parseJSON first attempts to use the browser's native JSON parser or a loaded library such as json2.js where applicable (which on a side note is the library the jQuery logic is based on if JSON is not defined). Thus jQuery can only be as permissive as that underlying implementation:

parseJSON: function( data ) {
    ...

    // Attempt to parse using the native JSON parser first
    if ( window.JSON && window.JSON.parse ) {
        return window.JSON.parse( data );
    }

    ...

    jQuery.error( "Invalid JSON: " + data );
},

As far as I know these implementations only adhere to the official JSON specification and do not accept single quotes, hence neither does jQuery.

Answer 2

If you need a single quote inside of a string, since \' is undefined by the spec, use \u0027 see http://www.utf8-chartable.de/ for all of them

edit: please excuse my misuse of the word backticks in the comments. I meant backslash. My point here is that in the event you have nested strings inside other strings, I think it can be more useful and readable to use unicode instead of lots of backslashes to escape a single quote. If you are not nested however it truly is easier to just put a plain old quote in there.

Answer 3

I understand where the problem lies and when I look at the specs its clear that unescaped single quotes should be parsed correctly.

I am using jquery`s jQuery.parseJSON function to parse the JSON string but still getting the parse error when there is a single quote in the data that is prepared with json_encode.

Could it be a mistake in my implementation that looks like this (PHP - server side):

$data = array();

$elem = array();
$elem['name'] = 'Erik';
$elem['position'] = 'PHP Programmer';
$data[] = json_encode($elem);

$elem = array();
$elem['name'] = 'Carl';
$elem['position'] = 'C Programmer';
$data[] = json_encode($elem);

$jsonString = "[" . implode(", ", $data) . "]";

The final step is that I store the JSON encoded string into an JS variable:

<script type="text/javascript">
employees = jQuery.parseJSON('<?=$marker; ?>');
</script>

If I use "" instead of '' it still throws an error.

SOLUTION:

The only thing that worked for me was to use bitmask JSON_HEX_APOS to convert the single quotes like this:

json_encode($tmp, JSON_HEX_APOS);

Is there another way of tackle this issue? Is my code wrong or poorly written?

Thanks

Answer 4

When You are sending a single quote in a query

empid = " T'via"
empid =escape(empid)

When You get the value including a single quote

var xxx  = request.QueryString("empid")
xxx= unscape(xxx)

If you want to search/ insert the value which includes a single quote in a query xxx=Replace(empid,"'","''")

Answer 5

Striking a similar issue using CakePHP to output a JavaScript script-block using PHP's native json_encode. $contractorCompanies contains values that have single quotation marks and as explained above and expected json_encode($contractorCompanies) doesn't escape them because its valid JSON.

<?php $this->Html->scriptBlock("var contractorCompanies = jQuery.parseJSON( '".(json_encode($contractorCompanies)."' );"); ?>

By adding addslashes() around the JSON encoded string you then escape the quotation marks allowing Cake / PHP to echo the correct javascript to the browser. JS errors disappear.

<?php $this->Html->scriptBlock("var contractorCompanies = jQuery.parseJSON( '".addslashes(json_encode($contractorCompanies))."' );"); ?>

Answer 6

I was trying to save a JSON object from a XHR request into a HTML5 data-* attribute. I tried many of above solutions with no success.

What I finally end up doing was replacing the single quote ' with it code ' using a regex after the stringify() method call the following way:

var productToString = JSON.stringify(productObject);
var quoteReplaced = productToString.replace(/'/g, "'");
var anchor = '<a data-product=\'' + quoteReplaced + '\' href=\'#\'>' + productObject.name + '</a>';
// Here you can use the "anchor" variable to update your DOM element.

Answer 7

Interesting. How are you generating your JSON on the server end? Are you using a library function (such as json_encode in PHP), or are you building the JSON string by hand?

The only thing that grabs my attention is the escape apostrophe (\'). Seeing as you're using double quotes, as you indeed should, there is no need to escape single quotes. I can't check if that is indeed the cause for your jQuery error, as I haven't updated to version 1.4.1 myself yet.