Adding Batch Upsert to MongoDB.

Question

It looks like MonogoDB does not support batch upsert. Is this Correct?

If not, how would you go about adding batch upsert with the existing API which is the most efficient? Does batch upsert even make sense for mongo db?

What exactly do you mean by 'batch upsert'? Adding documents that don't exist yet, while updating (replacing) objects that already exist? — mnemosyn, Jan 14 '12 at 13:31
yes but as one batch and NOT one by one! similar to the save method which does upsert for one document but it could do batch upsert for a batch of documents — iCode, Jan 15 '12 at 00:17
Does this thread provide you with assistance: http://stackoverflow.com/questions/5143756/upsert-multiple-records-with-mongodb — Bill, Jan 19 '12 at 18:19
This only works if the the same value is used for all the elements but I was more looking for something like Save method where you only change the object and the upsert method figure out will either create it or update it(not replacing it like save does). When I later on though about this, this may not make sense as monogo db has no idea what value of an existing item changed to update it properyl and it need to replace the whole thing (which probably takes much more time). — iCode, Jan 23 '12 at 23:31
Efficient all depends upon your needs, why are you looking to do such a massive upsert in an efficient manner? It kinda goes against the principles of data storage really, this should be confined to a mass update script which is only run once every so often etc etc — Sammaye, Jan 21 '13 at 19:56
https://jira.mongodb.org/browse/SERVER-4004?page=com.atlassian.jira.plugin.system.issuetabpanels%3Achangehistory-tabpanel hasn't been resovled — Ravi Khakhkhar, Jan 23 '13 at 14:43
@MSpreij This question is a year old, and the OP (iCode) seems to have been satisfied that "save" method would make most sense for their workflow. Would you like to elaborate on your use case? — mjhm, Jan 24 '13 at 17:31

score 2 · Answer 1 · answered May 22 '14 at 09:55

2

If you upgrade to MongoDB 2.6 you can take advantage of the new Bulk operations:

Bulk.find(<query>).upsert().update(<update>);
Bulk.find(<query>).upsert().updateOne(<update>);
Bulk.find(<query>).upsert().replaceOne(<replacement>);

answered May 22 '14 at 09:55

Vlad Mihalcea

142,745
71
566
911

score 1 · Answer 2 · answered Jan 28 '13 at 10:30

There is a utility called mongoimport that has an upsert flag. Something like

mongoimport -c myitems -d mydb --upsert items.json

Could this possibly achieve what you are looking for?

The default is to upsert based upon _id but you can change that using the --upsertFields flag documented here

http://docs.mongodb.org/manual/reference/mongoimport/#cmdoption-mongoimport--upsertFields

score 1 · Answer 3 · answered May 05 '15 at 15:18

For C# MongoDB.Driver I use next:

        var writeModels = new List<WriteModel<T>>();
        foreach (var entity in list)
        {
            var id = entity.Id;
            if (id == null)
            {
                writeModels.Add(new InsertOneModel<T>(entity));
            }
            else
            {
                var filter = new ExpressionFilterDefinition<T>(x => x.Id == id);
                var replaceModel = new ReplaceOneModel<T>(filter, entity);
                writeModels.Add(replaceModel);
            }
        }
        await getCollection().BulkWriteAsync(writeModels);

Adding Batch Upsert to MongoDB.

3 Answers3