MongoDB Bulk Silent Update

Question

I need to insert a set of documents in a remote MongoDB instance, in which some of these documents may already be residing at the instance i.e. having _id parameter. I want MongoDB to ignore such documents and commit the rest.

Currently the default behavior of PyMongo's insert method is that, it returns back a DuplicateError Exception for each of the duplicate document encountered at the remote server. I want to suppress this behaviour into a more fire and forget kind of functionality.

If anyone knows how to do this. It'll be appreciable.

http://stackoverflow.com/questions/2801008/mongodb-insert-if-not-exists did you look at that Q? — WiredPrairie, Jun 21 '13 at 11:00

score 1 · Accepted Answer · answered Jun 24 '13 at 13:07

For bulk inserts where there may be duplicate key errors you need to set the continue_on_error flag.

With this flag inserts will continue even if an error has happened. The last error will be reported back by getLastError - which you can catch or if you want fire and forget set the write concern to 0.

from pymongo import *
client = MongoClient()
coll = client.test.test
coll.drop()

# Add a test document
coll.save({'_id': 1, 'hello': 'world'})
print(coll.count())

# Without the flag - Boom
coll.insert([{"_id": 1, "hello": "world"}, 
             {"_id": 2, "Howdy": "Worldy"}])
print(coll.count())

# With a write concern of 0 - no error but not saved.
coll.insert([{"_id": 1, "hello": "world"}, 
             {"_id": 2, "Howdy": "Worldy"}], w=0)
print(coll.count())

# Will error but will insert as well
coll.insert([{"_id": 1, "hello": "world"}, 
             {"_id": 2, "Howdy": "Worldy"}], continue_on_error=True)
print(coll.count())

# With a continue_on_error and write concern of 0 - will only error if theres 
# some socket or network error
coll.insert([{"_id": 1, "hello": "world"}, 
             {"_id": 2, "Howdy": "Worldy"},
             {"_id": 3, "Hi": "World"}], w=0,  continue_on_error=True)
print(coll.count())

Thanks. I did figure it out myself also. Good to have a double check. — VaidAbhishek, Jun 24 '13 at 15:13

MongoDB Bulk Silent Update

1 Answers1