multi document insert using mongoengine into mongodb

Question

In my flask app I am using MongoeEgine. I am trying to insert multiple documents into my places collection in my MongoDB.

My document class is defined as

class places(db.Document):

  name = db.StringField(max_length=200, required=True)    
  loc = db.GeoPointField(required=True)

  def __unicode__(self):
    return self.name

    a=[]
    a.append({"name" : 'test' , "loc":[-87,101]})
    a.append({"name" : 'test' , "loc":[-88,101]})
    x= places(a)

The last statement fails

x= places(a)
TypeError: __init__() takes exactly 1 argument (2 given)

I also tried to save this to my instance

places.insert(x)
places.save(x)

both fail. Please help.

score 40 · Accepted Answer · answered Mar 01 '13 at 09:54

40

Places.objects.insert doesn't take a list of dictionaries it has to be Places instances. Normal operations would be to create individual instances of Places and save or insert eg:

Places(name="test", loc=[-87, 101]).save()
Places(name="test 2", loc=[-87, 101]).save()

However if you want to do a bulk insert you can pass a list of Places instances and call insert on the objects queryset eg:

Places.objects.insert([Places(name="test", loc=[-87, 101]), 
                       Places(name="test 2", loc=[-87, 101])])

answered Mar 01 '13 at 09:54

Ross

17,861
2
55
73

5

Thanks. How could i use this to do an upsert . I want to insert these rows if they are new , else update the existing row. Is there a Places.objects.upsert or Places.object.insert with upsert=True flag? – user1340513 Mar 03 '13 at 08:47
1

@Ross is there some simple way to avoid *NotUniqueError*? I read somewhere about using *ordered* in PyMongo bulk insert, but I would prefer to use Mongoengine. I know of _getCollection(), but still I would prefer to avoid not accessing the collection directly. I prefer using the Mongoengine syntax :) – giliev Nov 01 '16 at 23:48
1

@Ross Which one of the above operations is an efficient one i mean whether it is a save() operation or bulk insert operation. – Lohith Aug 16 '17 at 04:27
This does not work for me. I get an error even when bulk inserting into an empty DB. "mongoengine.errors.NotUniqueError: Document must not have _id value before bulk write (batch op errors occurred)". A comment in the mongoengine source says "inserting documents that already have an _id field will give huge performance debt or raise". I can't see how to get around this as the id attributes of the objects are None. – Michael Currin Jun 25 '19 at 19:01

x47 · Answer 2 · 2013-03-01T07:47:17.527

You try to initialize Document object for multiple documents at once. If you look at mongoengine's BaseDocument class, you'll see, that its __init__ method takes a dictionary of keyword arguments, which relate to fields of one single document.

If you want to do a bulk save, you have to make a list of places instances and pass it to insert() method.

a = []
a.append(places(**{"name": 'test', "loc": [-87,101]}))
a.append(places(**{"name": 'test', "loc": [-88,101]}))
x = places.objects.insert(a)

multi document insert using mongoengine into mongodb

2 Answers2

Linked