Parse nested json as raw string

Question

I am writing a Lambda function that receives an SQS queue object. The SQS sends a json object as a string value to SQS.

When I receive the request in the Lambda, AWS has wrapped it into a new JSON and since the json is a string value it becomes invalid json.

(Example) The json looks like:

{"uuid ":"someuuid"}

We send this as a string to the SQS. AWS then wraps this into something like:

{
  "Records": [
    {
      "messageId": "somemesasgeid",
      "receiptHandle": "MessageReceiptHandle",
      "body": {
          "Message":"{"uuid":"someUuid"}"
      },
      "attributes": {
        "ApproximateReceiveCount": "1",
        "SentTimestamp": "sometimestamp",
        "SenderId": "someid",
        "ApproximateFirstReceiveTimestamp": "sometimestamp"
      },
      "messageAttributes": {},
      "md5OfBody": "somebody",
      "eventSource": "aws:sqs",
      "eventSourceARN": "someARN",
      "awsRegion": "eu-west-1"
    }
  ]
}

Now the body.Message is not valid Json. I tried parsing it as a raw value like How can I include raw JSON in an object using Jackson? but it keeps complaining that it found a u where it was expecting a comma seperate object.

Since I can't post raw json to SQS and have to stringify it, how do I go about parsing this into an object where I can get the json message?

I tried creating a pojo and trying the above link, but jackson keeps complaining about the message variable.

--- update with code ---

private Response HandleServiceRequest(Map<String, Object> input) {

List<String> records = (List<String>) input.get("Records");
    for(String r : records) {
        SqsMessage m = objectMapper.readValue(r, SqsMessage.class);
    }
}

public class SqsMessage {
    // all other values
    SqsBody body;

// getters/setters
}

public class SqsBody {
// all other values
@JsonProperty("Message")
private Object message;
// getters/setters
@JsonRawValue
public String getMessage() {
   message == null ? null : message.toString();
}

public void setMessage(Object message){
this.message = message;
}
}

This is what I have now. I tried changing message to String but that did not change anything.

Could you try to treat that nested json as a string instead of json or a pojo? That way it should get escaped and result into something like `"Message":"{\"uuid\":\"someUuid\"}"` — Thomas, Jun 21 '19 at 09:50
I tried that. String and even Object. Jackson will still try to parse it. I could try to escape the json before sending it to SQS though.. Gonna try this — MrM, Jun 21 '19 at 09:53
Jackson shouldn't try to parse a string value unless you tell it that `"Message"` is something else. Can you provide more information, e.g. a [mcve]? — Thomas, Jun 21 '19 at 09:55
@Thomas This is what I have now. I tried changing the message to String but that made no difference to me. — MrM, Jun 21 '19 at 10:03
Well, you've posted how you're trying to _parse_ the json but what I was getting at is how do you _generate_ it? `"Message":"{"uuid":"someUuid"}"` is no valid json so Jackson will have a problem parsing that in any case. You need to have the nested json escaped upon generation, i.e. bring it to a form like I suggested in my first comment - only if the string content is properly escaped will Jackson be able to read the string as a whole. — Thomas, Jun 21 '19 at 10:10
The person who wrote the other end says he escapes it, he will double check. Will get back to you — MrM, Jun 21 '19 at 10:18
Similar in Python, but maybe the same works here: [How do I automatically fix an invalid JSON string?](https://stackoverflow.com/a/18515887/1639625). (I just tried, it works in Python for this example, and should work in Java, too, if the error messages are similar.) Of course, the preferred solution would be to not produce that invalid JSON in the first place. — tobias_k, Jun 21 '19 at 10:19
@Thomas So we rechecked and escaped them and that did indeed solve the problem. I can't accept your comment as answer though. — MrM, Jun 21 '19 at 10:36
@tobias_k Thanks for the link, it would have been a good last resort action but figuring out where the parsing was going wrong was indeed preferred. — MrM, Jun 21 '19 at 10:39

score 1 · Accepted Answer · answered Jun 21 '19 at 12:37

The basic problem is that "Message":"{"uuid":"someUuid"}" isn't valid json and thus Jackson cannot parse this part. From a json point of view the value of "Message" is "{" and thus the following uuid causes the error.

To handle that you'll need to properly escape the nested json so that it looks like any other string to Jackson, i.e. you'd want to get something like this:

"Message":"{\"uuid\":\"someUuid\"}"

Here the value of "Message" is "{\"uuid\":\"someUuid\"}" and thus Jackson can take that string and handle it further (put it into a string or unescape and parse).

That means you shouldn't try to tackle the problem at the parsing side but at the generation side, i.e. where the request body for HandleServiceRequest(...) is being created.

Parse nested json as raw string

1 Answers1