0

I'm working on this project where I have to perform mass insertion on MongoDB database

I understand that MongoDB is a document database and there is limit on size of each document as seen here

Now for mass insertion code look like this 

RockBand.collection.insert(mass_data)

mass_data is a array hash of like this 

[
  {
   name: "LedZepplin",
   member : 4, 
   studio_album : 10,
   ... 
   ...
   ...
  },
  { 
   name: "Linkin Park",
   member: 5,
   studio_album: 7,
   ...
   ... 
   ...
  },
  {
   ... 
   ...
   },
  ...
  ...
 ]

the total length of the array is 500K - 100K

an I knew for sure none of the above hash present in array which are basically a document in MongoDB are of size 16MB

So whenever I performer this 

RockBand.collection.insert(mass_data)

Why it keeping give me 16MB limit error as state above I'm quite sure that none of the above document persent in the array(i.e hash) does not weigh is of 16MB individual .

then why the error of data-size exceed for a document

Is it considering the whole array as single document when it should have be considering

each hash of the array as an individual document

Can Anyone Suggest

Btw I'm using Mongoid Driver on top of MongoDB ruby driver for connecting to MongoDB

Ratatouille
  • 1,372
  • 5
  • 23
  • 50

1 Answers1

0

When you insert an array like that, you are inserting the whole array as a single document. You have to insert each object in the array as with an individual insert command.

Philipp
  • 67,764
  • 9
  • 118
  • 153
  • so you mean Inserting an array would tell the mongo to consider the the whole thing in array as single document so how to achieve mass insertion then – Ratatouille Oct 09 '12 at 07:34
  • Just found a link on http://stackoverflow.com/questions/3772378/batch-insert-update-using-mongoid Here there doing nothing different is mention to what I doing for mass insertion – Ratatouille Oct 09 '12 at 07:40
  • I don't know ruby enough neither can I get a full picture of your source code, but can it be that you're actually storing a string rather than a ruby hash? Have you tried to the batch only, lets say 5 entries, rather than your whole batch. Maybe there's some problem within mongoid that it still has the 16mb limit on batches. – philnate Oct 09 '12 at 09:14
  • @phinate ,No I aint storing a ruby hash anyway If you wont I just paste the code over here – Ratatouille Oct 17 '12 at 12:03