0

Good Afternoon Everyone,

I have a question regarding upserting arrays to Mongo.

Let's use the following data as an example:

[{ 
  name: 'Charlie Brown',
  companies: ['xyz', 'abc'],
  data: {
    City: 'PHOENIX'
  } 
},
{ 
  name: 'Alex Brown',
  companies: ['xyz', 'ghj'],
  data: {
    State: 'GA'
  } 
},
{ 
  name: 'Charlie Brown',
  companies: ['abc', 'yui'],
  data: {
    City: 'FLINT',
    State: 'GA'
  } 
}]

The actual records are a bit more complex and more numerous than 3, but I hope this will serve sufficiently.

I want to insert/update/upsert these records into a mongoDB where matches (updates) do not overwrite the properties, but instead use $addToSet.

The end result should look like this:

[{ 
  name: 'Charlie Brown',
  companies: ['xyz', 'abc', 'yui'],
  data: {
    City: ['PHOENIX', 'FLINT'],
    STATE: 'GA'
  } 
},
{ 
  name: 'Alex Brown',
  companies: ['xyz', 'ghj'],
  data: {
    State: 'GA'
  } 
}]

I'm having a bit of trouble figuring out the correct mongo syntax for this. I know there is an upsert option on update, but how do I supply an array of records I would like to update, and then update individual records in the array?

I thought maybe the idea is to gather a set of names from the records, so

['Charlie Brown', 'Alex Brown']

and then use that set as a query field,

mongo.collection.update({
    ['Charlie Brown', 'Alex Brown'],
    {$addToSet:{companies: ??? what do I put here??}},
    {upsert: true}
})

What is the correct syntax (or correct method, perhaps using aggregations) to insert/update these records into Mongo?

Thank you!

Note: this question has similarities to Bulk upsert in MongoDB using mongoose , but the key here is also using the modifier $addToSet. Also, those solutions place the DB operation within a loop, is this normal with mongo?

Community
  • 1
  • 1
ZAR
  • 2,550
  • 4
  • 36
  • 66
  • 1
    The question you've linked has the correct approach. The loop is used to accumulate records and send them to server in batches. `$addToSet` is just another update operator. So in your case a sample record would be `mongo.collection.update({'Charlie Brown'}, {$addToSet:{companies: ['yui']}}, {upsert: true})` and you prepare other records in the similar way and add them to the batch and send these batches to server. – s7vr Apr 06 '17 at 17:15
  • @Veeram, so it is perfectly acceptable to got to the database to find, process, update within a loop of tens of thousands of records? I guess I was just taught to never place in database operations within loops. – ZAR Apr 06 '17 at 17:24
  • 1
    Is database operation running inside loop ? Yes. Is it calling database for each loop iteration ? No. It hits database once after it accumulates 1000 records. I guess the alternative will be managing it using promises async instead of doing it synchronously. Nevertheless you have to somehow send this batches to server. So it comes down to memory vs i/o. Read the comments and other answers from the linked question. Will probably help you. – s7vr Apr 06 '17 at 17:39
  • Got it, thanks @Veeram – ZAR Apr 06 '17 at 17:51

0 Answers0