2

New to play,scala, and reactivemongo and the documentation is not very noob friendly.

I see the Bulk Insert section at See Bulk Insert

but I don't know why they aren't showing it contained in a method? I am expecting a request with JSON data containing multiple objects in it. How do I set up a bulk insert that handles multiple inserts with errors that can be returned.

For example by single insert method is as follows:

def createFromJson = Action(parse.json) {

request =>
  try {
    val person = request.body.validate[Person].get

    val mongoResult = Await.result(collection.insert(person),Duration.apply(20,"seconds"))
    if(mongoResult.hasErrors) throw new Exception(mongoResult.errmsg.getOrElse("something unknown"))


    Created(Json.toJson(person))
}
catch {
    case e: Exception => BadRequest(e.getMessage)
}

}

Joan
  • 4,079
  • 2
  • 28
  • 37
Jeff
  • 211
  • 3
  • 11

3 Answers3

3

Here is a full example how you can do it:

class ExampleController @Inject()(database: DefaultDB) extends Controller {

  case class Person(firstName: String, lastName: String)

  val personCollection: BSONCollection = database.collection("persons")
  implicit val PersonJsonReader: Reads[Person] = Json.reads[Person]
  implicit val PersonSeqJsonReader: Reads[Seq[Person]] = Reads.seq(PersonJsonReader)
  implicit val PersonJsonWriter: Writes[Person] = Json.writes[Person]
  implicit val PersonSeqJsonWriter: Writes[Seq[Person]] = Writes.seq(PersonJsonWriter)
  implicit val PersonBsonWriter = Macros.writer[Person]

  def insertMultiple = Action.async(parse.json) { implicit request =>
    val validationResult: JsResult[Seq[Person]] = request.body.validate[Seq[Person]]

    validationResult.fold(
      invalidValidationResult => Future.successful(BadRequest),
      // [1]
      validValidationResult => {
        val bulkDocs = validValidationResult.
          map(implicitly[personCollection.ImplicitlyDocumentProducer](_))

        personCollection.bulkInsert(ordered = true)(bulkDocs: _*).map {
          case insertResult if insertResult.ok =>
            Created(Json.toJson(validationResult.get))
          case insertResult => 
            InternalServerError
        }
      }
    )
  }
}

The meat of it all sits in the lines after [1]. validValidationResult is a variable of type Seq[Person] and contains valid data at this point. Thats what we want to insert into the database.

To do that we need to prepare the documents by mapping each document through the ImplicitlyDocumentProducer of your target collection (here personCollection). Thats leaves you with bulkDocs of type Seq[personCollection.ImplicitlyDocumentProducer]. You can just use bulkInsert() with that:

personCollection.bulkInsert(ordered = true)(bulkDocs: _*)

We use _* here to splat the Seq since bulkInsert() expects varargs and not a Seq. See this thread for more info about it. And thats basically it already.

The remaing code is handling play results and validating the received request body to make sure it contains valid data.

Here are a few general tips to work with play/reactivemongo/scala/futures:

Avoid Await.result. You basically never need it in production code. The idea behind futures is to perform non-blocking operations. Making them blocking again with Await.result defeats the purpose. It can be useful for debugging or test code, but even then there are usually better ways to go about things. Scala futures (unlike java ones) are very powerful and you can do a lot with them, see e.g. flatMap/map/filter/foreach/.. in the Future scaladoc. The above code for instance makes use of exactly that. It uses Action.async instead of Action at the controller method. This means it has to return a Future[Result] instead of a Result. Which is great because ReactiveMongo returns a bunch of Futures for all operations. So all you have to do is execute bulkInsert, which returns a Future and use map() to map the returned Future[MultiBulkWriteResult] to a Future[Result]. This results in no blocking and play can work with the returned future just fine.

Of course the above example can be improved a bit, I tried to keep it simple. For instance you should return proper error messages when returning BadRequest (request body validation failed) or InternalServerError (database write failed). You can get more info about the errors from invalidValidationResult and insertResult. And you could use Formats instead of that many Reads/Writes (and also use them for ReactiveMongo). Check the play json documentation as well as the reactive mongo doc for more info on that.

Community
  • 1
  • 1
alextsc
  • 1,368
  • 1
  • 8
  • 13
  • Thanks it definitely cleared some things up for me. Do you know of any resources to future my understanding. I come from a MEAN mvc background and all this converting from JSON to BSON and readers and futures are completely new to me, along with the play scala syntax. – Jeff Mar 14 '16 at 21:46
  • Yeah its a lot at first. I suggest you go through it slowly and check the documentation for every class or method/function you dont know. The [play docs](https://www.playframework.com/documentation/2.5.x/ScalaHome) are quite good, for reads/writes check the chapter about json. ReactiveMongo also has one for the [BSON equivalent](http://reactivemongo.org/releases/0.11/documentation/bson/typeclasses.html). For futures check [this blogpost](http://danielwestheide.com/blog/2013/01/09/the-neophytes-guide-to-scala-part-8-welcome-to-the-future.html) for instance. – alextsc Mar 14 '16 at 21:53
  • Note that I'm being "lazy" in my example code above, I use macro functions provided by both play and reactive mongo to generate the reads/writes. They can alternatively be written by hand. Don't get confused when you see that. It's the same thing and you should practice doing it "manually" first. – alextsc Mar 14 '16 at 21:56
3

Although the previous answer is correct. We can reduce the boilerplate using JSONCollection

package controllers

import javax.inject._

import play.api.libs.json._
import play.api.mvc._
import play.modules.reactivemongo._
import reactivemongo.play.json.collection.{JSONCollection, _}
import utils.Errors

import scala.concurrent.{ExecutionContext, Future}


case class Person(name: String, age: Int)

object Person {
  implicit val formatter = Json.format[Person]
}

@Singleton
class PersonBulkController @Inject()(val reactiveMongoApi: ReactiveMongoApi)(implicit exec: ExecutionContext) extends Controller with MongoController with ReactiveMongoComponents {

  val persons: JSONCollection = db.collection[JSONCollection]("person")

  def createBulkFromJson = Action.async(parse.json) { request =>

    Json.fromJson[Seq[Person]](request.body) match {
      case JsSuccess(newPersons, _) =>
        val documents = newPersons.map(implicitly[persons.ImplicitlyDocumentProducer](_))

        persons.bulkInsert(ordered = true)(documents: _*).map{ multiResult =>
          Created(s"Created ${multiResult.n} persons")
        }

      case JsError(errors) =>
        Future.successful(BadRequest("Could not build an array of persons from the json provided. " + errors))
    }
  }
}

In build.sbt

libraryDependencies ++= Seq(
  "org.reactivemongo" %% "play2-reactivemongo" % "0.11.12"
)

Tested with play 2.5.1 although it should compile in previous versions of play.

Jonas Anso
  • 2,057
  • 14
  • 13
  • 1
    using `persons` as a JSONCollection results in `ImplicitlyDocumentProducer` not to be found. However when I changed the definition of `persons` to a BSONCollection, `ImplicitlyDocumentProducer` was found but `(_)` produced a type mismatch where `models.Person` was found but `PersonController.this.personCollection.ImplicitlyDocumentProducer` was required. – Jeff Mar 16 '16 at 17:12
  • Review your imports. Specially collection._ – Jonas Anso Mar 16 '16 at 18:20
  • The complete code snippet compiles and inserts the data. – Jonas Anso Mar 16 '16 at 18:20
  • @JonasAnso tried your snippet (copy paste) with reactive mongo 0.11.12 and playfw 2.4.6. Got compilation error as mentioned by Jeff. What versions do you use? I also cant figure out how to fix this issue. – kurochenko Jun 07 '16 at 08:43
  • 1
    I have updated response @kurochenko adding the dependencies. Hope it helps. You can also take a look at https://github.com/jonasanso/play-reactive-mongo-db This exact code is not there but you can find in CityController.createBulkFromJson something very similar – Jonas Anso Jun 07 '16 at 20:44
  • Is there any way to get bulk insert working without having that "Person" class but a BSONDocument ? – Daniel Jul 04 '16 at 16:44
  • Yes. Maybe this helps you https://github.com/ReactiveMongo/ReactiveMongo/blob/master/driver/src/test/scala/CommonUseCases.scala In case you have problems maybe you can create a new question and me or others can help. – Jonas Anso Jul 04 '16 at 20:18
  • I had also the "type mismatch" issue with the ImplicitlyDocumentProducer, I fixed this by adding the corresponding object with the implicit Json formatter. `object Person {` `implicit val formatter = Json.format[FuelStation]` `}` – Benoit Feb 13 '17 at 14:02
  • for json `implicit val formatter = Json.format[FuelStation]` or for bson : `implicit val writer = Macros.writer[Person]` – Benoit Feb 13 '17 at 14:31
  • Thanks @benoit Not sure what FuelStation means, probably is something in your code. I imagine you meant ```object Person { implicit val formatter = Json.format[Person] }``` but that code is already in the response. How could I improve the response so other people do not have the same "type mismatch" issue? – Jonas Anso Feb 13 '17 at 20:01
  • I confirm, FuelStation should to be changed to Person – Benoit Feb 15 '17 at 13:38
1

FYI, as previous answers said, there are two ways to manipulate JSON data: use ReactiveMongo module + Play JSON library, or use ReactiveMongo's BSON library.

The documentation of ReactiveMongo module for Play Framework is available online. You can find code examples there.

Roger Chien
  • 344
  • 3
  • 13