Parsing JSON data with unicode

Question

My JSON string looks like this:

"{\"TrackingInformationResponse\": {\u000a  \"shipments\": [\u000a    {\u000a      \"shipmentId\": \"03015035146308\",\u000a      \"uri\": \"\/ntt-service-rest\/api\/shipment\/03015035146308\/0\",\u000a      \"assessedNumberOfItems\": 1,\u000a      \"deliveryDate\": \"2013-05-13T11:47:00\",\u000a      \"estimatedTimeOfArrival\": \"2013-05-13T16:00:00\",\u000a      \"service\": {\u000a        \"code\": \"88\",\u000a        \"name\": \"DPD\"\u000a      },\u000a      \"consignor\": {\u000a        \"name\": \"Webhallen Danmark ApS\",\u000a        \"address\": {\u000a          \"street1\": \"Elsa Brändströms Gata 52\",\u000a          \"city\": \"HÄGERSTEN\",\u000a          \"countryCode\": \"SWE\",\u000a          \"country\": \"Sverige\",\u000a          \"postCode\": \"12952\"\u000a        }\u000a      },\u000a      \"consignee\": {\u000a        \"name\": \"Lene Bjerre Kontor & IT Service\",\u000a        \"address\": {\u000a          \"street1\": \"Lene Bjerre\",\u000a          \"street2\": \"Ørbækvej 8, Hoven\",\u000a          \"city\": \"TARM\",\u000a          \"countryCode\": \"???\",\u000a          \"postCode\": \"6880\"\u000a        }\u000a      },\u000a      \"statusText\": {\u000a        \"header\": \"Forsendelsen er udleveret\",\u000a        \"body\": \"Forsendelsen blev leveret 13-05-2013 kl. 11:47\"\u000a      },\u000a      \"status\": \"DELIVERED\",\u000a      \"totalWeight\": {\u000a        \"value\": \"0.55\",\u000a        \"unit\": \"kg\"\u000a      },\u000a      \"totalVolume\": {\u000a        \"value\": \"0.005\",\u000a        \"unit\": \"m3\"\u000a      },\u000a      \"items\": [\u000a        {\u000a          \"itemId\": \"03015035146308\",\u000a          \"dropOffDate\": \"2013-05-08T17:18:00\",\u000a          \"deliveryDate\": \"2013-05-13T11:47:00\",\u000a          \"status\": \"DELIVERED\",\u000a          \"statusText\": {\u000a            \"header\": \"Forsendelsen er udleveret til modtageren\",\u000a            \"body\": \"Forsendelsen blev udleveret  13-05-2013 kl. 11:47\"\u000a          },\u000a          \"statedMeasurement\": {\u000a            \"weight\": {\u000a              \"value\": \"0.55\",\u000a              \"unit\": \"kg\"\u000a            },\u000a            \"length\": {\u000a              \"value\": \"0.28\",\u000a              \"unit\": \"m\"\u000a            },\u000a            \"height\": {\u000a              \"value\": \"0.1\",\u000a              \"unit\": \"m\"\u000a            },\u000a            \"width\": {\u000a              \"value\": \"0.2\",\u000a              \"unit\": \"m\"\u000a            },\u000a            \"volume\": {\u000a              \"value\": \"0.005\",\u000a              \"unit\": \"m3\"\u000a            }\u000a          },\u000a          \"events\": [\u000a            {\u000a              \"eventTime\": \"2013-05-08T16:42:00\",\u000a              \"eventCode\": \"68\",\u000a              \"eventDescription\": \"Oplysning om forsendelsen er modtaget\",\u000a              \"location\": {\u000a                \"displayName\": \"Webhallen Danmark ApS\",\u000a                \"name\": \"Webhallen Danmark ApS\",\u000a                \"countryCode\": \"SWE\",\u000a                \"country\": \"Sverige\"\u000a              }\u000a            },\u000a            {\u000a              \"eventTime\": \"2013-05-08T17:18:00\",\u000a              \"eventCode\": \"74\",\u000a              \"eventDescription\": \"Transporten af forsendelsen er påbegyndt\",\u000a              \"location\": {\u000a                \"locationId\": \"126\",\u000a                \"displayName\": \"Segeltorp\",\u000a                \"name\": \"Segeltorp\",\u000a                \"countryCode\": \"SWE\",\u000a                \"country\": \"Sverige\",\u000a                \"postcode\": \"14172\",\u000a                \"city\": \"Segeltorp\",\u000a                \"locationType\": \"HUB\"\u000a              }\u000a            },\u000a            {\u000a              \"eventTime\": \"2013-05-08T17:18:00\",\u000a              \"eventCode\": \"31\",\u000a              \"eventDescription\": \"Forsendelsen er på vej\",\u000a              \"location\": {\u000a                \"locationId\": \"126\",\u000a                \"displayName\": \"Segeltorp\",\u000a                \"name\": \"Segeltorp\",\u000a                \"countryCode\": \"SWE\",\u000a                \"country\": \"Sverige\",\u000a                \"postcode\": \"14172\",\u000a                \"city\": \"Segeltorp\",\u000a                \"locationType\": \"HUB\"\u000a              }\u000a            },\u000a            {\u000a              \"eventTime\": \"2013-05-10T04:34:00\",\u000a              \"eventCode\": \"31\",\u000a              \"eventDescription\": \"Forsendelsen er på vej\",\u000a              \"location\": {\u000a                \"locationId\": \"201\",\u000a                \"displayName\": \"Malmö\",\u000a                \"name\": \"Malmö\",\u000a                \"countryCode\": \"SWE\",\u000a                \"country\": \"Sverige\",\u000a                \"postcode\": \"21241\",\u000a                \"city\": \"Malmö\",\u000a                \"locationType\": \"HUB\"\u000a              }\u000a            },\u000a            {\u000a              \"eventTime\": \"2013-05-10T22:46:13\",\u000a              \"eventCode\": \"31\",\u000a              \"eventDescription\": \"Forsendelsen er på vej\",\u000a              \"location\": {\u000a                \"locationId\": \"0707\",\u000a                \"displayName\": \"Kolding Pakkecenter\",\u000a                \"name\": \"Kolding Pakkecenter\",\u000a                \"countryCode\": \"DNK\",\u000a                \"country\": \"Danmark\",\u000a                \"postcode\": \"2605\",\u000a                \"city\": \"Brøndby\",\u000a                \"locationType\": \"HUB\"\u000a              }\u000a            },\u000a            {\u000a              \"eventTime\": \"2013-05-11T01:05:00\",\u000a              \"eventCode\": \"31\",\u000a              \"eventDescription\": \"Forsendelsen er på vej\",\u000a              \"location\": {\u000a                \"locationId\": \"0050\",\u000a                \"displayName\": \"Kolding\",\u000a                \"name\": \"Kolding\",\u000a                \"countryCode\": \"DNK\",\u000a                \"country\": \"Danmark\",\u000a                \"locationType\": \"DPD_DEPOT\"\u000a              }\u000a            },\u000a            {\u000a              \"eventTime\": \"2013-05-13T07:59:00\",\u000a              \"eventCode\": \"113\",\u000a              \"eventDescription\": \"Forsendelsen gøres klar til levering\",\u000a              \"location\": {\u000a                \"locationId\": \"0506\",\u000a                \"displayName\": \"Herning\",\u000a                \"name\": \"Herning\",\u000a                \"countryCode\": \"DNK\",\u000a                \"country\": \"Danmark\",\u000a                \"locationType\": \"DPD_DEPOT\"\u000a              }\u000a            },\u000a            {\u000a              \"eventTime\": \"2013-05-13T07:59:00\",\u000a              \"eventCode\": \"31\",\u000a              \"eventDescription\": \"Forsendelsen er på vej\",\u000a              \"location\": {\u000a                \"locationId\": \"0506\",\u000a                \"displayName\": \"Herning\",\u000a                \"name\": \"Herning\",\u000a                \"countryCode\": \"DNK\",\u000a                \"country\": \"Danmark\",\u000a                \"locationType\": \"DPD_DEPOT\"\u000a              }\u000a            },\u000a            {\u000a              \"eventTime\": \"2013-05-13T07:59:44\",\u000a              \"eventCode\": \"z65\",\u000a              \"eventDescription\": \"Forsendelsen er ankommet distributionscenter\",\u000a              \"location\": {\u000a                \"locationId\": \"6900\",\u000a                \"displayName\": \"Skjern\",\u000a                \"name\": \"Skjern Distributionscenter\",\u000a                \"countryCode\": \"DNK\",\u000a                \"country\": \"Danmark\",\u000a                \"postcode\": \"6900\",\u000a                \"city\": \"Skjern\",\u000a                \"locationType\": \"DEPOT\"\u000a              }\u000a            },\u000a            {\u000a              \"eventTime\": \"2013-05-13T11:47:00\",\u000a              \"eventCode\": \"21\",\u000a              \"eventDescription\": \"Forsendelsen er udleveret\",\u000a              \"location\": {\u000a                \"locationId\": \"0506\",\u000a                \"displayName\": \"Herning\",\u000a                \"name\": \"Herning\",\u000a                \"countryCode\": \"DNK\",\u000a                \"country\": \"Danmark\",\u000a                \"locationType\": \"DPD_DEPOT\"\u000a              }\u000a            }\u000a          ],\u000a          \"references\": [\u000a            {\u000a              \"value\": \"5008074\",\u000a              \"type\": \"CU\",\u000a              \"name\": \"Afsenderreference \"\u000a            }\u000a          ]\u000a        }\u000a      ],\u000a      \"additionalServices\": [\u000a      ],\u000a      \"splitStatuses\": [\u000a      ],\u000a      \"shipmentReferences\": [\u000a        {\u000a          \"value\": \"5008074\",\u000a          \"type\": \"CU\",\u000a          \"name\": \"Afsenderreference \"\u000a        }\u000a      ]\u000a    }\u000a  ]\u000a}}"

I am getting this data from a server using the following code:

    public String readJSONFeed(String URL) {
    StringBuilder stringBuilder = new StringBuilder();
    HttpClient httpClient = new DefaultHttpClient();
    HttpGet httpGet = new HttpGet(URL);
    try {
        HttpResponse response = httpClient.execute(httpGet);
        StatusLine statusLine = response.getStatusLine();
        int statusCode = statusLine.getStatusCode();
        if (statusCode == 200) {
            HttpEntity entity = response.getEntity();
            InputStream inputStream = entity.getContent();
            BufferedReader reader = new BufferedReader(
                    new InputStreamReader(inputStream, "UTF-8"));
            String line;
            while ((line = reader.readLine()) != null) {
                stringBuilder.append(line);
            }
            inputStream.close();
        } else {
            Log.d("JSON", "Failed to download file");
        }
    } catch (Exception e) {
        Log.d("readJSONFeed", e.getLocalizedMessage());
    }        
    return stringBuilder.toString();
}

When I try to parse the string with

JSONObject jsonObject = new JSONObject(result);

, I get an exception, most likely because the JSON is invalid. If I try to manually replace the unicode characters like this, it works perfectly:

\u000a is replaced with a new line
\" is replaced with just "
First and last " is removed

How would I go about parsing this data?

I think this might help you : http://stackoverflow.com/questions/3537706/howto-unescape-a-java-string-literal-in-java/4298836#4298836 — reidzeibel, May 28 '13 at 11:14

score 0 · Answer 1 · answered May 28 '13 at 14:46

"{\"TrackingInformationResponse\": {\u000a  \"shipments\": [\u000a...

You have a JSON string that is itself wrapped in a string literal format. It's not 100% sure what the wrapping string format is because there are many formats that use \u and \" escapes - you should get whatever documentation you can on the service you are calling to try to figure out what it's supposed to be. It would seem to be strange and unnecessary to wrap a web service return result in a string literal format like this.

One possibility is Java string literal; in that case see unescapeJava in Apache's StringEscapeUtils.

However probably the most likely would be JSON string literal - ie double-JSON-encoding. In this case you'd have to JSON-decode the string once before looking at the nested JSON structure inside it.

Assuming you are using org.json though, there is not a direct JSONString parser to go with the JSONObject and JSONArray parsers. However you could always wrap your string in a structure that one of those will understand, eg:

String unwrapped= new JSONArray("["+result+"]").get(0);
JSONObject o= new JSONObject(unwrapped);

I can't find any information on the data itself. Would it be possible to manually replace all \u000a's with new lines etc? — Lazze, May 28 '13 at 18:43
@Lazze: unlikely - there may be special characters other than `\u000A` which you don't have in your example. I would expect to see a real backslash encoded as double-backslash, for example. And you have `\/` in there for forward slash which is a puzzle as neither Java nor JSON literal formats require that escape (though both allow it). I would go with a JSON decoder until such time as you can get any definitive answer from the provider of the service you are connecting to. — bobince, May 31 '13 at 09:04

Parsing JSON data with unicode

1 Answers1