Efficient POJO mapping to/from Java Mongo DBObject using Jackson

Question

Although similar to Convert DBObject to a POJO using MongoDB Java Driver my question is different in that I am specifically interested in using Jackson for mapping.

I have an object which I want to convert to a Mongo DBObject instance. I want to use the Jackson JSON framework to do the job.

One way to do so is:

DBObject dbo = (DBObject)JSON.parse(m_objectMapper.writeValueAsString(entity));

However, according to https://github.com/FasterXML/jackson-docs/wiki/Presentation:-Jackson-Performance this is the worst way to go. So, I am looking for an alternative. Ideally, I would like to be able to hook into the JSON generation pipeline and populate a DBObject instance on the fly. This is possible, because the target in my case is a BasicDBObject instance, which implements the Map interface. So, it should fit into the pipeline easily.

Now, I know I can convert an object to Map using the ObjectMapper.convertValue function and then recursively convert the map to a BasicDBObject instance using the map constructor of the BasicDBObject type. But, I want to know if I can eliminate the intermediate map and create the BasicDBObject directly.

Note, that because a BasicDBObject is essentially a map, the opposite conversion, namely from a scalar DBObject to a POJO is trivial and should be quite efficient:

DBObject dbo = getDBO();
Class clazz = getObjectClass();
Object pojo = m_objectMapper.convertValue(dbo, clazz);

Lastly, my POJO do not have any JSON annotations and I would like it to keep this way.

Answer 1

You can probably use Mixin annotations to annotate your POJO and the BasicDBObject (or DBObject), so annotations is not a problem. Since BasicDBOject is a map, you can use @JsonAnySetter on the put method.

m_objectMapper.addMixInAnnotations(YourMixIn.class, BasicDBObject.class);

public interface YourMixIn.class {
    @JsonAnySetter
    void put(String key, Object value);
}

This is all I can come up with since I have zero experience with MongoDB Object.

Update: MixIn are basically a Jackson mechanism to add annotation to a class without modifying said class. This is a perfect fit when you don't have control over the class you want to marshal (like when it's from an external jar) or when you don't want to clutter your classes with annotation.

In your case here, you said that BasicDBObject implements the Map interface, so that class has the method put, as defined by the map interface. By adding @JsonAnySetter to that method, you tell Jackson that whenever he finds a property that he doesn't know after introspection of the class to use the method to insert the property to the object. The key is the name of the property and the value is, well, the value of the property.

All this combined makes the intermediate map go away, since Jackson will directly convert to the BasicDBOject because it now knows how to deserialize that class from Json. With that configuration, you can do:

DBObject dbo = m_objectMapper.convertValue(pojo, BasicDBObject.class);

Note that I haven't tested this because I don't work with MongoDB, so there might be some loose ends. However, I have used the same mechanism for similar use cases without any problem. YMMV depending on the classes.

Answer 2

Here's an example of a simple serializer (written in Scala) from POJO to BsonDocument which could be used with version 3 of Mongo driver. The de-serializer would be somewhat more difficult to write.

Create a BsonObjectGenerator object which would do a streaming serialization to Mongo Bson directly:

val generator = new BsonObjectGenerator
mapper.writeValue(generator, POJO)
generator.result()

Here's the code for a serializer:

class BsonObjectGenerator extends JsonGenerator {

  sealed trait MongoJsonStreamContext extends JsonStreamContext

  case class MongoRoot(root: BsonDocument = BsonDocument()) extends MongoJsonStreamContext {
    _type = JsonStreamContext.TYPE_ROOT

    override def getCurrentName: String = null

    override def getParent: MongoJsonStreamContext = null
  }

  case class MongoArray(parent: MongoJsonStreamContext, arr: BsonArray = BsonArray()) extends MongoJsonStreamContext {
    _type = JsonStreamContext.TYPE_ARRAY

    override def getCurrentName: String = null

    override def getParent: MongoJsonStreamContext = parent
  }

  case class MongoObject(name: String, parent: MongoJsonStreamContext, obj: BsonDocument = BsonDocument()) extends MongoJsonStreamContext {
    _type = JsonStreamContext.TYPE_OBJECT

    override def getCurrentName: String = name

    override def getParent: MongoJsonStreamContext = parent
  }

  private val root = MongoRoot()
  private var node: MongoJsonStreamContext = root

  private var fieldName: String = _

  def result(): BsonDocument = root.root

  private def unsupported(): Nothing = throw new UnsupportedOperationException

  override def disable(f: Feature): JsonGenerator = this

  override def writeStartArray(): Unit = {
    val array = new BsonArray
    node match {
      case MongoRoot(o) =>
        o.append(fieldName, array)
        fieldName = null
      case MongoArray(_, a) =>
        a.add(array)
      case MongoObject(_, _, o) =>
        o.append(fieldName, array)
        fieldName = null
    }
    node = MongoArray(node, array)
  }

  private def writeBsonValue(value: BsonValue): Unit = node match {
    case MongoRoot(o) =>
      o.append(fieldName, value)
      fieldName = null
    case MongoArray(_, a) =>
      a.add(value)
    case MongoObject(_, _, o) =>
      o.append(fieldName, value)
      fieldName = null
  }

  private def writeBsonString(text: String): Unit = {
    writeBsonValue(BsonString(text))
  }

  override def writeString(text: String): Unit = writeBsonString(text)

  override def writeString(text: Array[Char], offset: Int, len: Int): Unit = writeBsonString(new String(text, offset, len))

  override def writeString(text: SerializableString): Unit = writeBsonString(text.getValue)

  private def writeBsonFieldName(name: String): Unit = {
    fieldName = name
  }

  override def writeFieldName(name: String): Unit = writeBsonFieldName(name)

  override def writeFieldName(name: SerializableString): Unit = writeBsonFieldName(name.getValue)

  override def setCodec(oc: ObjectCodec): JsonGenerator = this

  override def useDefaultPrettyPrinter(): JsonGenerator = this

  override def getFeatureMask: Int = 0

  private def writeBsonBinary(data: Array[Byte]): Unit = {
    writeBsonValue(BsonBinary(data))
  }

  override def writeBinary(bv: Base64Variant, data: Array[Byte], offset: Int, len: Int): Unit = {
    val res = if (offset != 0 || len != data.length) {
      val subset = new Array[Byte](len)
      System.arraycopy(data, offset, subset, 0, len)
      subset
    } else {
      data
    }
    writeBsonBinary(res)
  }

  override def writeBinary(bv: Base64Variant, data: InputStream, dataLength: Int): Int = unsupported()

  override def isEnabled(f: Feature): Boolean = false

  override def writeRawUTF8String(text: Array[Byte], offset: Int, length: Int): Unit = writeBsonString(new String(text, offset, length, "UTF-8"))

  override def writeRaw(text: String): Unit = unsupported()

  override def writeRaw(text: String, offset: Int, len: Int): Unit = unsupported()

  override def writeRaw(text: Array[Char], offset: Int, len: Int): Unit = unsupported()

  override def writeRaw(c: Char): Unit = unsupported()

  override def flush(): Unit = ()

  override def writeRawValue(text: String): Unit = writeBsonString(text)

  override def writeRawValue(text: String, offset: Int, len: Int): Unit = writeBsonString(text.substring(offset, offset + len))

  override def writeRawValue(text: Array[Char], offset: Int, len: Int): Unit = writeBsonString(new String(text, offset, len))

  override def writeBoolean(state: Boolean): Unit = {
    writeBsonValue(BsonBoolean(state))
  }

  override def writeStartObject(): Unit = {
    node = node match {
      case p@MongoRoot(o) =>
        MongoObject(null, p, o)
      case p@MongoArray(_, a) =>
        val doc = new BsonDocument
        a.add(doc)
        MongoObject(null, p, doc)
      case p@MongoObject(_, _, o) =>
        val doc = new BsonDocument
        val f = fieldName
        o.append(f, doc)
        fieldName = null
        MongoObject(f, p, doc)
    }
  }

  override def writeObject(pojo: scala.Any): Unit = unsupported()

  override def enable(f: Feature): JsonGenerator = this

  override def writeEndArray(): Unit = {
    node = node match {
      case MongoRoot(_) => unsupported()
      case MongoArray(p, a) => p
      case MongoObject(_, _, _) => unsupported()
    }
  }

  override def writeUTF8String(text: Array[Byte], offset: Int, length: Int): Unit = writeBsonString(new String(text, offset, length, "UTF-8"))

  override def close(): Unit = ()

  override def writeTree(rootNode: TreeNode): Unit = unsupported()

  override def setFeatureMask(values: Int): JsonGenerator = this

  override def isClosed: Boolean = unsupported()

  override def writeNull(): Unit = {
    writeBsonValue(BsonNull())
  }

  override def writeNumber(v: Int): Unit = {
    writeBsonValue(BsonInt32(v))
  }

  override def writeNumber(v: Long): Unit = {
    writeBsonValue(BsonInt64(v))
  }

  override def writeNumber(v: BigInteger): Unit = unsupported()

  override def writeNumber(v: Double): Unit = {
    writeBsonValue(BsonDouble(v))
  }

  override def writeNumber(v: Float): Unit = {
    writeBsonValue(BsonDouble(v))
  }

  override def writeNumber(v: BigDecimal): Unit = unsupported()

  override def writeNumber(encodedValue: String): Unit = unsupported()

  override def version(): Version = unsupported()

  override def getCodec: ObjectCodec = unsupported()

  override def getOutputContext: JsonStreamContext = node

  override def writeEndObject(): Unit = {
    node = node match {
      case p@MongoRoot(_) => p
      case MongoArray(p, a) => unsupported()
      case MongoObject(_, p, _) => p
    }
  }
}

Answer 3

You might be intereted in checking how jongo does it. It is open source and the code can be found on github. Or you could also simply use their library. I use a mix of jongo and plain DBObjects when I need more flexibility.

They claim that they are (almost) as fast as using the Java driver directly so I suppose their method is efficient.

I use the little helper utility class below which is inspired from their code base and uses a mix of Jongo (the MongoBsonFactory) and Jackson to convert between DBObjects and POJOs. Note that the getDbObject method does a deep copy of the DBObject to make it editable - if you don't need to customise anything you can remove that part and improve performance.

import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.ObjectReader;
import com.fasterxml.jackson.databind.ObjectWriter;
import com.fasterxml.jackson.databind.introspect.VisibilityChecker;
import com.mongodb.BasicDBObject;
import com.mongodb.DBEncoder;
import com.mongodb.DBObject;
import com.mongodb.DefaultDBEncoder;
import com.mongodb.LazyWriteableDBObject;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import org.bson.LazyBSONCallback;
import org.bson.io.BasicOutputBuffer;
import org.bson.io.OutputBuffer;
import org.jongo.marshall.jackson.bson4jackson.MongoBsonFactory;

public class JongoUtils {

    private final static ObjectMapper mapper = new ObjectMapper(MongoBsonFactory.createFactory());

    static {
        mapper.setVisibilityChecker(VisibilityChecker.Std.defaultInstance().withFieldVisibility(
                JsonAutoDetect.Visibility.ANY));
    }

    public static DBObject getDbObject(Object o) throws IOException {
        ObjectWriter writer = mapper.writer();
        ByteArrayOutputStream baos = new ByteArrayOutputStream();

        writer.writeValue(baos, o);
        DBObject dbo = new LazyWriteableDBObject(baos.toByteArray(), new LazyBSONCallback());
        //turn it into a proper DBObject otherwise it can't be edited.
        DBObject result = new BasicDBObject();
        result.putAll(dbo);
        return result;
    }

    public static <T> T getPojo(DBObject o, Class<T> clazz) throws IOException {
        ObjectReader reader = mapper.reader(clazz);
        DBEncoder dbEncoder = DefaultDBEncoder.FACTORY.create();
        OutputBuffer buffer = new BasicOutputBuffer();
        dbEncoder.writeObject(buffer, o);

        T pojo = reader.readValue(buffer.toByteArray());

        return pojo;
    }
}

Sample usage:

Pojo pojo = new Pojo(...);
DBObject o = JongoUtils.getDbObject(pojo);
//you can customise it if you want:
o.put("_id", pojo.getId());

Answer 4

I understand that this is a very old question, but if asked today I would instead recommend the built-in POJO support on the official Mongo Java driver.

Answer 5

Here's an update to assylias' answer that doesn't require Jongo and is compatible with the Mongo 3.x drivers. It also handles nested object graphs, I couldn't get that to work with LazyWritableDBObject which has been removed in the mongo 3.x drivers anyway.

The idea is to tell Jackson how to serialize an object to a BSON byte array, and then deserialize the BSON byte array into BasicDBObject. I'm sure you can find some low level API in the mongo-java-drivers if you want to ship the BSON bytes directly to the database. You will need a dependency to bson4jackson in order for ObjectMapper to serialize BSON when you call writeValues(ByteArrayOutputStream, Object):

import com.fasterxml.jackson.databind.ObjectMapper;
import com.mongodb.BasicDBObject;
import com.mongodb.DBObject;
import de.undercouch.bson4jackson.BsonFactory;
import de.undercouch.bson4jackson.BsonParser;
import org.bson.BSON;
import org.bson.BSONObject;

import java.io.ByteArrayOutputStream;
import java.io.IOException;

public class MongoUtils {

    private static ObjectMapper mapper;

    static {
        BsonFactory bsonFactory = new BsonFactory();
        bsonFactory.enable(BsonParser.Feature.HONOR_DOCUMENT_LENGTH);
        mapper = new ObjectMapper(bsonFactory);
    }

    public static DBObject getDbObject(Object o) {
        try {
            ByteArrayOutputStream baos = new ByteArrayOutputStream();
            mapper.writeValue(baos, o);

            BSONObject decode = BSON.decode(baos.toByteArray());
            return new BasicDBObject(decode.toMap());
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }
}