Converting DynamoDB JSON to Standard JSON with Java

Question

I need to convert a AWS DYNAMODB JSON to a standard JSON object. so I can remove the data type from the DynamoDB JSON Something more like:

in DYNAMODB JSON:

"videos": [
    {
      "file": {
        "S": "file1.mp4"
      },
      "id": {
        "S": "1"
      },
      "canvas": {
        "S": "This is Canvas1"
      }
    },
    {
      "file": {
        "S": "main.mp4"
      },
      "id": {
        "S": "0"
      },
      "canvas": {
        "S": "this is a canvas"
      }
    }
  ]

to Standard JSON
 "videos": [
    {
      "file": "file1.mp4"
      ,
      "id": "1"
      ,
      "canvas":  "This is Canvas1"
      ,
      "file": "main.mp4"
      ,
      "id":  "0"
      ,
      "canvas": "this is a canvas"

    }
  ]

I found a nice tool in Javascript but is there any tool in Java in order to do that?

Answer 1

You can use ItemUtils class in aws sdk. Below is sample code using Kotlin:

import com.amazonaws.services.dynamodbv2.document.ItemUtils
import com.amazonaws.services.dynamodbv2.model.AttributeValue

fun main(args: Array<String>) {
    val data = HashMap<String,AttributeValue>()
    data.put("hello",AttributeValue().withS("world"))
    println(data.toString())
    println(ItemUtils.toItem(data).toJSON())
}

Output:

{hello={S: world,}}
{"hello":"world"}

Answer 2

Below is the complete code for converting from Dynamo JSON to Standard JSON:

import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.internal.InternalUtils;
import com.amazonaws.services.dynamodbv2.model.AttributeValue;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.amazonaws.services.lambda.runtime.events.DynamodbEvent;
import com.amazonaws.services.lambda.runtime.events.DynamodbEvent.DynamodbStreamRecord;
import com.google.gson.Gson;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;

/**
 * Main Lambda class to receive event stream, parse it to Survey
 * and process them.
 */
public class SurveyEventProcessor implements
        RequestHandler<DynamodbEvent, String> {

    private static final String INSERT = "INSERT";

    private static final String MODIFY = "MODIFY";

    public String handleRequest(DynamodbEvent ddbEvent, Context context) {

        List<Item> listOfItem = new ArrayList<>();
        List<Map<String, AttributeValue>> listOfMaps = null;
        for (DynamodbStreamRecord record : ddbEvent.getRecords()) {

            if (INSERT.equals(record.getEventName()) || MODIFY.equals(record.getEventName())) {
                listOfMaps = new ArrayList<Map<String, AttributeValue>>();
                listOfMaps.add(record.getDynamodb().getNewImage());
                listOfItem = InternalUtils.toItemList(listOfMaps);
            }

            System.out.println(listOfItem);
            try {
               // String json = new ObjectMapper().writeValueAsString(listOfItem.get(0));
                Gson gson = new Gson();
                Item item = listOfItem.get(0);

                String json = gson.toJson(item.asMap());
                System.out.println("JSON is ");
                System.out.println(json);
            }catch (Exception e){
                e.printStackTrace();
            }
        }


        return "Successfully processed " + ddbEvent.getRecords().size() + " records.";
    }
} 

Answer 3

Following is a simple solution which can be applied to convert any DynamoDB Json to Simple JSON.

//passing the reponse.getItems() 
public static Object getJson(List<Map<String,AttributeValue>> mapList) {
    List<Object> finalJson= new ArrayList();
    for(Map<String,AttributeValue> eachEntry : mapList) {
        finalJson.add(mapToJson(eachEntry));
    }
    return finalJson;
}


//if the map is null then it add the key and value(string) in the finalKeyValueMap
public static Map<String,Object> mapToJson(Map<String,AttributeValue> keyValueMap){
    Map<String,Object> finalKeyValueMap = new HashMap();
    for(Map.Entry<String, AttributeValue> entry : keyValueMap.entrySet()) 
    {
        if(entry.getValue().getM() == null) {
            finalKeyValueMap.put(entry.getKey(),entry.getValue().getS());
        }
        else {
            finalKeyValueMap.put(entry.getKey(),mapToJson(entry.getValue().getM()));
        }
    }
    return finalKeyValueMap;
}

This will produce your desired Json in the form of List>. Then using the Gson you can convert it into the Json format.

Gson gson = new Gson();
String jsonString = gson.toJson(getJson(response.getItems()));

Answer 4

Convert your JSON string to a Map first using

JSONObject jsonObj = new JSONObject(logString);
HashMap<String, Object> myMap = new Gson().fromJson(jsonObj.toString(), HashMap.class);

Function for converting DynamoDB JSON to Standard JSON

public static Map<String,Object> mapToJson(Map map){
        Map<String,Object> finalKeyValueMap = new HashMap();
        Iterator it = map.entrySet().iterator();

        while(it.hasNext()) {
            Map.Entry pair = (Map.Entry)it.next();

            Map obj1 = (Map) pair.getValue();
            if(obj1.get("m") == null) {
                if(obj1.get("n") != null)
                    finalKeyValueMap.put(pair.getKey().toString(),obj1.get("n"));
                else if(obj1.get("s") != null)
                    finalKeyValueMap.put(pair.getKey().toString(),obj1.get("s"));
            }
            else {
                Map obj2 = (Map) pair.getValue();
                Map obj3 = (Map) obj2.get("m");
                finalKeyValueMap.put(pair.getKey().toString(),mapToJson(obj3));
            }

        }
        System.out.println(finalKeyValueMap.toString());
        return finalKeyValueMap;
    }

calling mapToJson(myMap); will return a standard Map which you can convert back to JSON

Answer 5

Amazon has an internal utility that can do the core of the job in 1 line.

Map<String, Object> jsonMapWithId = InternalUtils.toSimpleMapValue(attributeValueMap);

A long version which contains some configurations:

import com.amazonaws.services.dynamodbv2.document.internal.InternalUtils;
import com.amazonaws.services.dynamodbv2.model.AttributeValue;
import java.lang.Map;
import java.lang.HashMap;

public class JsonConvert {
    public static void main(String[] args) {
        Map<String,Object> jsonMap = new HashMap();
        jsonMap.put("string", "foo");
        jsonMap.put("number", 123);
        Map<String,Object> innerMap = new HashMap();
        innerMap.put("key", "value");
        jsonMap.put("map", innerMap);
        AttributeValue attributeValue = InternalUtils.toAttributeValue(jsonMap);
        Map<String, AttributeValue> attributeValueMap = new HashMap();
        attributeValueMap.put("id", attributeValue);

        Map<String, Object> jsonMapWithId = InternalUtils.toSimpleMapValue(attributeValueMap);
        Map<String, Object> jsonMap = jsonMapWithId.get("id"); // This is a regular map, values are not Amazon AttributeValue

        Gson gson = new Gson(); 
        String json = gson.toJson(jsonMap); 

        System.out.println(json);
    }
}

Answer 6

There is now EnhancedDocument interface to address this https://aws.amazon.com/blogs/devops/introducing-the-enhanced-document-api-for-dynamodb-in-the-aws-sdk-for-java-2-x/

Answer 7

import com.amazonaws.services.dynamodbv2.document.ItemUtils;
import com.amazonaws.services.dynamodbv2.model.AttributeValue;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.MapperFeature;
import com.fasterxml.jackson.databind.ObjectMapper;

import java.io.IOException;
import java.util.Map;

public final class Utility {

  public static <T> String convertDynamoDbJsonToNormalJson(final JsonNode node, final Class<T> type)
      throws IOException {
    final var objectMapper =
        new ObjectMapper().configure(MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES, true);

    final Map<String, AttributeValue> itemMap =
        objectMapper.convertValue(node, new TypeReference<>() {});
    final var item = ItemUtils.toItem(itemMap);
    final var asPojo = objectMapper.readValue(item.toJSON(), type);
    return objectMapper.writeValueAsString((Object) asPojo);
  }
}