1

I have questions on InsertManyAsync vs BulkWriteAsync via the NuGet below:

https://www.nuget.org/packages/MongoDB.Driver/2.12.3?_src=template

I want to export 300,000 rows of data around 20 MB, convert them into JSON and import them into Mongo Altas.

My questions:

1 Which operation, InsertManyAsync vs BulkWriteAsync, is transactional, e.g. all or nothing?

2 What is the maximum rows or size allowed for each operation?

The link below or elsewhere don't have the answers:

MongoDB C# driver 2.0 InsertManyAsync vs BulkWriteAsync

https://mongodb.github.io/mongo-csharp-driver/2.12/apidocs/html/M_MongoDB_Driver_IMongoCollection_1_BulkWrite.htm

https://mongodb.github.io/mongo-csharp-driver/2.12/apidocs/html/Overload_MongoDB_Driver_IMongoCollection_1_InsertManyAsync.htm

Pingpong
  • 7,681
  • 21
  • 83
  • 209

1 Answers1

2

there is no difference between InsertManyAsync and BulkWriteAsync other than BulkWriteAsync can work not only with insert. In other words, InsertManyAsync calls BulkWriteAsync internally.

Is the operation is one transactional operation, e.g. all or nothing?

see transactions. Also, see ordered/unordered option here.

What is the maximum rows or size allowed?

I don't think that there is any limitation other then restriction on the document size which is 16 MB. Pay attention that bulk items can be merged into a single document before sending to the server (that should not be bigger 16 MB in sum) or they should be sent separately one by one. This logic depends on whether you use isOrdered option and whether all items in your batch are with the same bulk type.

dododo
  • 3,872
  • 1
  • 14
  • 37
  • regarding limits, just noticed this doc https://docs.mongodb.com/manual/reference/limits/#mongodb-limit-Write-Command-Batch-Limit-Size, but i didn't use it in practice – dododo Jun 07 '21 at 17:14
  • For `InsertMany`: "100,000 writes are allowed in a single batch operation", this makes the data importing more complex than `BulkWriteAsync`. We have 300,000 rows less than 16MB, so it seems bulk import is easier to code. Please correct me if I am wrong. – Pingpong Jun 08 '21 at 15:21
  • if you use the same bulkType for all items in `BulkWriteAsync` you should not see any difference, because `InsertManyAsync` calls `BulkWriteAsync` almost immediately. See this line https://github.com/mongodb/mongo-csharp-driver/blob/master/src/MongoDB.Driver/MongoCollectionBase.cs#L541 – dododo Jun 08 '21 at 16:49
  • "100,000 writes are allowed in a single batch operation", which method does that statement apply to? – Pingpong Jun 08 '21 at 17:51
  • Does `InsertManyAsync` and `BulkWriteAsync` has the batch limitation? – Pingpong Jun 08 '21 at 17:56