Pymongo insert_many BulkWriteError

Question

I am trying to insert the following list of dictionaries named posts to mongo, and got a BulkWriteError: batch op errors occurred error which I don't know how to fix.

posts:

[{'#AUTHID': 'fffafe151f07a30a0ede2038a897b680',
  'Records': [
   {'DATE': '07/22/09 05:54 PM',
    'STATUS': 'Is flying back friday night in time to move the rest of his stuff then go to work the next morning... great.'},
    ......

   {'DATE': '07/19/09 04:39 PM', 'STATUS': 'is stealing his net by the lake'}]},

 {'#AUTHID': 'fffafe151f07a30a0ede2038a897b680',
  'Records': [
   {'DATE': '07/22/09 05:54 PM',
    'STATUS': 'Is flying back friday night in time to move the rest of his stuff then go to work the next morning... great.'},
   {'DATE': '07/19/09 04:39 PM', 'STATUS': 'is stealing his net by the lake'},
  ....

The code I used:

collection = db.posts
collection.insert_many(p for p in posts )

But then I got an error that says BulkWriteError: batch op errors occurred and only managed to import the first dictionary (corresponding to the first #AUTHID)

I found a link that describes similar situation but it doesn't explain much about why this happens or how to solve this issue. It's under the _Why does PyMongo add an id field to all of my documents? in the following link:
https://github.com/mongodb/mongo-python-driver/blob/master/doc/faq.rst#id25

Based on your sample `posts` above, it seems like the document is duplicated because the content is the same. Please note that the FAQ says about inserting **single document** as multiple. I suspect your `dict(s)` is the same instance. — CallMeLaNN, Feb 22 '17 at 11:15

CallMeLaNN · Answer 1 · 2017-02-22T11:18:46.770

Not to late to answer here, you almost there. I am not sure if the FAQ updated but please read it properly:

when calling insert_many() with a list of references to a single document raises BulkWriteError

Note that it says single or in other word, same instance. The example in the FAQ shows how to produce the error with the same instance. You can check if it is the same by using id() to display the memory address. In fact, I can see the content of your documents is the same. Most probably (but not necessarily) it is the same instance.

print id(posts[0])
print id(posts[1])

If any of the dict having the same instance, then something wrong during preparing the posts variable. Just make sure all list items should have different instance because you are inserting (many) different documents!

ghanshyam dudhatra · Answer 2 · 2016-07-14T05:50:29.363

Here is output in this output records are store which are in list.

from pymongo import MongoClient 
client = MongoClient('localhost', 27017)
db = client['post']
posts = [{'#AUTHID': 'fffafe151f07a30a0ede2038a897b680',
    'Records': [
        {'DATE': '07/22/09 05:54 PM',
            'STATUS': 'Is flying back friday night in time to move the rest of his stuff then go to work the next morning... great.'},


        {'DATE': '07/19/09 04:39 PM', 'STATUS': 'is stealing his net by the lake'}]},

    {'#AUTHID': 'fffafe151f07a30a0ede2038a897b680',
        'Records': [
        {'DATE': '07/22/09 05:54 PM',
        'STATUS': 'Is flying back friday night in time to move the rest of his stuff then go to work the next morning... great.'},
        {'DATE': '07/19/09 04:39 PM', 'STATUS': 'is stealing his net by the lake'}]}]
collection = db.posti.insert_many(p for p in posts )

Pymongo insert_many BulkWriteError

2 Answers2

Linked