0

I have a problem with double quotes. We have a JAVA web server where a string input is parsed into JSON. Sometimes the data given to us includes quotes within quotes which breaks the JSON parser. It seems it is not possible for them to escape these characters, so can I escape them on our end?

For example the data is like this: {"id":"101","user":"username","comment":"Example: "Comment: 123" test" }

The quotes inside "comment" are what I need to escape.

Erdem KESLI
  • 1
  • 1
  • 4
    "It seems it is not possible for them to escape these characters" - it's unclear what you mean by this. If the input is meant to be JSON but isn't valid JSON, I'd strongly recommend pushing back on whoever is providing the input. trying to work around it by guessing how to fix the brokenness is likely to be less effective than getting the source of the problem fixed. – Jon Skeet Oct 21 '21 at 13:29
  • For how it *should* be escaped in the input, see https://stackoverflow.com/questions/15637429/how-to-escape-double-quotes-in-json - see also https://www.json.org/json-en.html There should be a backslash before double quotes that are intended to be part of a string. – Hulk Oct 21 '21 at 13:50
  • Whether you can reliably fix this on your side depends on how consistent this behavior is, and how acceptable it is to occasionally get it wrong - "repair" heuristics are susceptible to all kinds of subtle bugs. If the behavior ever changes, this will break things (e.g. they fix this bug, and add another field after the comment...). – Hulk Oct 21 '21 at 14:06
  • Ther is also a current [related question](https://stackoverflow.com/questions/69661082/how-to-escape-double-quotes-in-java-json) - too much escaping instead of too little, but the answer stays the same - it's best to ensure your input is correct, if possible. – Hulk Oct 21 '21 at 14:19
  • the old topics are not same or similar to my case, they can be used before sent to data, but I need a solution after I get the data – Erdem KESLI Oct 21 '21 at 14:27

1 Answers1

1

Sometimes the data given to us includes quotes within quotes which breaks the JSON parser.

Then it is NOT proper JSON, period. Those quotes within quotes must be escaped in the JSON.

It seems it is not possible for them to escape these characters ...

More likely, someone doesn't want to invest the effort to fix the broken software that is generating invalid JSON.

.... so can I escape them on our end?

You won't be escaping at your end. You will be writing a custom JSON parser that copes with certain kinds of brokenness in it input data, and hopefully corrects it to match what the sender intends. It will be effectively guessing where the escapes should have been in the test being parsed.

It will be a lot of work. Have you ever written a parser before?

One of the difficulties will be knowing where the quoted string ought to end.

Compare these:

{"id":"101","user":"username","comment":"Example: "Comment: 123" test" }

{"id":"101","user":"username","comment":"Example: "Comment: 123" test" }" }

In the first case you might expect the "comment" attribute's value to end at test. But what about the second case? Is the } part of the attribute value? Why wasn't in in the first case?

The problem is that when you have syntactically incorrect JSON, a correcting parser could be corrected it in many different ways to end up with valid JSON. Knowing which correction is the correct correction is ... umm ... a problem.

I think you should follow Jon Skeet's advice.

Push back.

Tell them to fix the broken JSON at their end.

Tell them that the requirements say >>JSON<< and stuff that doesn't parse according to the standard JSON grammar is NOT JSON.

If they insist:

  • Bill them for the extra work that is beyond the scope of the agreed requirements; i.e. JSON.

  • Point out that it may be impossible to write a parser for their stuff that will always interpret their "not JSON" text according to the expectations of the authors. (It depends on how complex the JSON schema is and how wide-spread the incorrect quoting problem is.)

Stephen C
  • 698,415
  • 94
  • 811
  • 1,216