13

I want to aggregate and insert the results into an existing collection, without deleting that collection. The documentation seems to suggest that this isn't directly possible. I find that hard to believe.

The map-reduce functionality has 'output modes', including 'merge', which does what I want. I'm looking for the equivalent for aggregation.

The new $out aggregation stage supports inserting into a collection, but it replaces the collection rather than updating it. If I did this I would (I think) have to run another map-reduce to merge this into another collection, which seems inefficient.

Am I missing something or is the functionality just missing from the aggregation feature?

Joe
  • 46,419
  • 33
  • 155
  • 245

4 Answers4

11

I used the output from aggregation to insert/merge to collection:

    db.coll2.insert(
      db.coll1.aggregate([]).toArray()
    )
NBjerre
  • 151
  • 1
  • 5
4

Reading the documentation answers this question quite precisely. Atm mongo is not able to do what you want.

The $out operation creates a new collection in the current database if one does not already exist. The collection is not visible until the aggregation completes. If the aggregation fails, MongoDB does not create the collection.

If the collection specified by the $out operation already exists, then upon completion of aggregation the $out stage atomically replaces the existing collection with the new results collection. The $out operation does not change any indexes that existed on the previous collection. If the aggregation fails, the $out operation makes no changes to the previous collection.

Community
  • 1
  • 1
chk
  • 534
  • 3
  • 12
  • Thanks, I did read that page (hence the link in my question) and described the work-around. I was asking if there was some other mechanism for doing this. – Joe Jan 07 '14 at 16:26
  • if you're working directly in the shell you could consider something like the solution being described here http://learnmongo.com/posts/easily-move-documents-between-collections-or-databases/ but of course this will not be executed on the server side but requires fetching and re-inserting the data – chk Jan 07 '14 at 16:28
  • Thanks, but this would result in copying millions of records back and forward, which far exceeds 16MB! – Joe Jan 07 '14 at 16:32
  • yeah I was quite sure this is not suitable for you. Your work-around is probably the way to go. I guess sth like 'mongoimport' is also no option? – chk Jan 07 '14 at 16:35
2

For anyone coming to this more recently, this is available from version 4.2, you will be able to do this using the $merge operator in an aggregation pipeline. It needs to be the last stage in the pipeline.

{ $merge: { into: "myOutput", on: "_id", whenMatched: "replace", whenNotMatched: "insert" } }
rrrr-o
  • 2,447
  • 2
  • 24
  • 52
0

If your not stuck on using the Aggregation operators, you could do an incremental map-reduce on the collection. This operator allows you to merge results into an existing collection.

See documentation below:

http://docs.mongodb.org/manual/tutorial/perform-incremental-map-reduce/

jrajaratnam
  • 1
  • 1