How to force JSON-lib's JSONObject.put(..) to escape a string containing JSON?

Question

When using JSON-lib's JSONObject, how can I stop the put method from storing a String which contains JSON as JSON rather than as an escaped string?

For instance:

JSONObject obj = new JSONObject();
obj.put("jsonStringValue","{\"hello\":\"world\"}");
obj.put("naturalStringValue", "\"hello world\"");
System.out.println(obj.toString());
System.out.println(obj.getString("jsonStringValue"));
System.out.println(obj.getString("naturalStringValue"));

prints:

{"jsonStringValue":{"hello":"world"},"naturalStringValue":"\"hello world\""}
{"hello":"world"}
"hello world"

and I want it to print:

{"jsonStringValue":"{\"hello\":\"world\"}","naturalStringValue":"\"hello world\""}
{"hello":"world"}
"hello world"

Yes, I realize this is obnoxious. However, this is in support of a JSON serialization pipeline for which, for interoperability's sake, this is the expected behavior. There are cases in which we would be serializing user input which may be/contain valid JSON. We wouldn't want the user input to become a part of the JSON object that we're serializing said input to.

Manual escaping doesn't work because it causes JSON-lib to escape the \ characters:

JSONObject obj = new JSONObject();
obj.put("naturalJSONValue","{\"hello\":\"world\"}");
obj.put("escapedJSONValue", "{\\\"hello\\\":\\\"world\\\"}");
System.out.println(obj.toString());
System.out.println(obj.getString("naturalJSONValue"));
System.out.println(obj.getString("escapedJSONValue"));

Output:

{"naturalJSONValue":{"hello":"world"},"escapedJSONValue":"{\\\"hello\\\":\\\"world\\\"}"}
{"hello":"world"}
{\"hello\":\"world\"}

At this point, any workarounds to enable manual selective escaping of a complex JSON object would completely negate the value of using JSON-lib in the first place.

Also, I understand that this question has been asked before, but unfortunately I cannot accept its answer so easily. JSON-lib is a heavily-used dependency in many areas of my project and swapping it out would be a big undertaking. I need to be absolutely sure that there's no way to achieve this goal with JSON-lib before I can entertain a swap to Jackson, simple-json, or Gson.

score 5 · Answer 1 · edited Nov 24 '21 at 14:19

5

This worked for me with json-lib 2.4:

System.out.println(
    new JSONStringer()
        .object()
            .key("jsonStringValue")
                .value("{\"hello\":\"world\"}")
            .key("naturalStringValue")
                .value("\"hello world\"")
        .endObject()
    .toString());

The output is:

{"jsonStringValue":"{\"hello\":\"world\"}","naturalStringValue":"\"hello world\""}

edited Nov 24 '21 at 14:19

E_net4

27,810
13
101
139

answered May 30 '11 at 03:41

marsbear

1,459
10
23

I would consider it, but per the interoperability requirement this serialization pipeline must be compatible with an existing system for which this is a somewhat common use case. – Ben Burns May 30 '11 at 03:43
It is a specific existing legacy system with which this one must interface. The system transports untrusted JSON as a properly escaped string rather than as part of the object itself. I need to replicate this behavior. – Ben Burns May 30 '11 at 16:18
Well then what about my previous suggestion but applied directly after json-encoding your stuff? Means: Pre-escape the special fields, json-lib encode everything and finally post-deescape those special fields. But that's quite an effort tho not that hard to achieve with regex. – marsbear May 30 '11 at 16:30
Per the question "At this point, any workarounds to enable manual selective escaping of a complex JSON object would completely negate the value of using JSON-lib in the first place." -- especially if said manual escaping takes place post object serialization. I'd rather deal with swapping out the library at that point. – Ben Burns May 30 '11 at 17:36
Also, it's not that easy to achieve with regex. For security's sake this needs to be a guaranteed solution that doesn't fall susceptible to weird edge cases, and this type of escaping must only be performed on a specific field in the JSON object. – Ben Burns May 30 '11 at 17:39
I thought so but you said that replacing json-lib is very hard too as your environment depends on it. I'm sorry but I am out of crazy ideas :/ Well one last one: Have you thought about patching json-lib? – marsbear May 30 '11 at 19:35
I'd be happy to write a patch to JSON-lib and submit it, but if it isn't accepted I don't want to maintain it myself. Swapping it out will be a pain, but it's the right thing to do over hack-like workarounds or maintaining my own patch to it. – Ben Burns May 30 '11 at 19:43
I do appreciate your effort, though. I think this is just an intractable problem. – Ben Burns May 30 '11 at 19:49
Found a solution. I am still looking for a way to achieve the same result with JSONObject, hang tight :) – marsbear May 30 '11 at 21:27
Found no easy way tih JSONObject so far but JsonBuilder and JsonStringer seem to work fine. I modified my answer – marsbear May 31 '11 at 13:51

score 1 · Accepted Answer · edited Jun 20 '20 at 09:12

Use single quotes to quote the string. From the documentation:

Strings may be quoted with ' (single quote).

Strings do not need to be quoted at all if they do not begin with a quote or single quote, and if they do not contain leading or trailing spaces, and if they do not contain any of these characters: { } [ ] / \ : , = ; # and if they do not look like numbers and if they are not the reserved words true, false, or null.

So modifying your example:

net.sf.json.JSONObject obj = new net.sf.json.JSONObject();
obj.put("jsonStringValue","{\"hello\":\"world\"}");
obj.put("quotedJsonStringValue","\'{\"hello\":\"world\"}\'");
obj.put("naturalStringValue", "\"hello world\"");
System.out.println(obj.toString());
System.out.println(obj.getString("jsonStringValue"));
System.out.println(obj.getString("quotedJsonStringValue"));
System.out.println(obj.getString("naturalStringValue"));

Produces:

{"jsonStringValue":{"hello":"world"},"quotedJsonStringValue":"{\"hello\":\"world\"}","naturalStringValue":"\"hello world\""}
{"hello":"world"}
{"hello":"world"}
"hello world"

Note how quotedJsonStringValue has been treated as a string value and not JSON, and appears quoted in the output JSON.

As far as I understood the op the field-content is created by users and may contain anything including quotes of any kind and / or json. Would that work with your example, too? — marsbear, Jun 03 '11 at 17:22
@marsbear, if the data could include single-quotes, then yup, they would need to be escaped or stripped. Using `object().key().value()` would definitely be easier, I just chimed in because @Ben seemed to need to a workaround with just `JSONObject`. — Paul V, Jun 03 '11 at 21:22

How to force JSON-lib's JSONObject.put(..) to escape a string containing JSON?

2 Answers2

Linked