0

Given a large (millions+) collection of documents similar to:

{ _id : ObjectId, "a" : 3, "b" : 5 }

What is the most efficient way to process these documents directly on the server, with the results added to each document within the same collection? For example, add a key c whose value equals a+b.

{ _id : ObjectId, "a" : 3, "b" : 5, "c" : 8 }

I'd prefer to do this in the shell.

Seems that find().forEach() would waste time in transit between the db and the shell, and mapReduce() seems intended to process groups of objects down into aggregated data (though I may be misunderstanding).

EDIT: I'd prefer a solution that doesn't block, if there is one (other than using a cursor on the client)...

mongotop
  • 7,114
  • 14
  • 51
  • 76
ericsoco
  • 24,913
  • 29
  • 97
  • 127
  • Seems I could do this with db.eval(): http://www.mongodb.org/display/DOCS/Server-side+Code+Execution#Server-sideCodeExecution-Using%7B%7Bdb.eval%28%29%7D%7D, but that will be blocking. I thought perhaps update() would work, but it doesn't seem to be possible to reference the iterated document from within an update(): http://stackoverflow.com/questions/3788256/mongodb-updating-documents-using-data-from-the-same-document. So, doesn't seem to be a way to do this without blocking except a forEach on a client-side cursor... – ericsoco Sep 11 '12 at 19:03

2 Answers2

2

From the MongoDB Docs on db.eval():

"db.eval() is used to evaluate a function (written in JavaScript) at the database server. This is useful if you need to touch a lot of data lightly. In that scenario, network transfer of the data could be a bottleneck."

The documentation has an example of how to use it that is very similar to what you are trying to do.

Shaun
  • 284
  • 2
  • 3
  • right, as i wrote in my comment above it looks like db.eval() is a possibility, but it's blocking. didn't mention i'd prefer not to block... – ericsoco Sep 11 '12 at 21:04
1

forEach is your best option. I would run it on the server (from the shell) to reduce latency.

Eve Freeman
  • 32,467
  • 4
  • 86
  • 101