3

Desired Behaviour

Pull a range of objects from an array of objects and push them back to the array at a new index.

For example, pull objects from the array where their index is between 0 and 2, and push them back to the array at position 6.

For reference, in jQuery, the desired behaviour can be achieved with:

if (before_or_after === "before") {
    $("li").eq(new_position).before($("li").slice(range_start, range_end + 1));
} else if (before_or_after === "after") {
    $("li").eq(new_position).after($("li").slice(range_start, range_end + 1));
}

jsFiddle demonstration

Schema

{
    "_id": ObjectId("*********"),
    "title": "title text",
    "description": "description text",
    "statements": [
    {
        "text": "string",
        "id": "********"
    },
    {
        "text": "string",
        "id": "********"
    },
    {
        "text": "string",
        "id": "********"
    },
    {
        "text": "string",
        "id": "********"
    },
    {
        "text": "string",
        "id": "********"
    }]
}

What I've Tried

I am able to reposition a single object in an array of objects with the code below.

It uses pull to remove the object from the array and push to add it back to the array at a new position.

In order to do the same for a range of objects, I think I just need to modify the $pull and $push variables but:

  • I can't figure out how to use $slice in this context, either as a projection or an aggregation, in a $pull operation
  • Because I can't figure out the first bit, I don't know how to attempt the second bit - the $push operation
// define the topic_id to search for  
var topic_id = request_body.topic_id;

// make it usable as a search query  
var o_id = new ObjectID(topic_id);

// define the statement_id to search for   
var statement_id = request_body.statement_id;

// define new position
var new_position = Number(request_body.new_position);

// define old position
var old_position = Number(request_body.old_position);

// define before or after (this will be relevant later)
// var before_or_after = request_body.before_or_after;

// define the filter 
var filter = { _id: o_id };

// define the pull update - to remove the object from the array of objects  
var pull_update = {
    $pull: {
        statements: { id: statement_id }  // <----- how do i pull a range of objects here  
    }
};

// define the projection so that only the 'statements' array is returned
var options = { projection: { statements: 1 } };

try {
    // perform the pull update  
    var topic = await collection.findOneAndUpdate(filter, pull_update, options);

    // get the returned statement object so that it can be inserted at the desired index
    var returned_statement = topic.value.statements[old_position];

    // define the push update - to add the object back to the array at the desired position
    var push_update = {
        $push: {
            statements: {
                $each: [returned_statement],
                $position: new_position
            }
        }     // <----- how do i push the range of objects back into the array here  
    };

    // perform the push update  
    var topic = await collection.findOneAndUpdate(filter, push_update);

}

Environments

##### local
  
$  mongod --version
db version v4.0.3

$  npm view mongodb version
3.5.9

$ node -v
v10.16.3

$ systeminfo
OS Name:        Microsoft Windows 10 Home
OS Version:     10.0.18363 N/A Build 18363

##### production
  
$ mongod --version
db version v3.6.3

$ npm view mongodb version
3.5.9

$ node -v
v8.11.4

RedHat OpenShift Online, Linux

Edit

Gradually, figuring out parts of the problem, I think:

Using the example here, the following returns objects from array with index 0 - 2 (ie 3 objects):

db.topics.aggregate([
    { "$match": { "_id": ObjectId("********") } },
    { "$project": { "statements": { "$slice": ["$statements", 0, 3] }, _id: 0 } }
])

Not sure how to use that in a pull yet...

I also looked into using $in (even though i would prefer to just grab a range of objects than have to specify each object's id), but realised it does not preserve the order of the array values provided in the results returned:

Does MongoDB's $in clause guarantee order

Here is one solution to re-ordering results from $in in Node:

https://stackoverflow.com/a/34751295

user1063287
  • 10,265
  • 25
  • 122
  • 218
  • 1
    Are you able to upgrade to MongoDB v4.2? If so you can perform the update in one operation, using [updates with aggregation pipeline](https://docs.mongodb.com/manual/tutorial/update-documents-with-aggregation-pipeline/). Otherwise I *think* you'll have to get the whole document and process it in node.js and update with the modified array. – thammada.ts Jul 28 '20 at 14:12
  • i'm waiting to hear back on whether 4.0 and then 4.2 is available in production server so i can start the upgrade path from 3.6.3, bit concerned there may be compatibility bugs introduced from the upgrade, but looks like its the only option. – user1063287 Jul 28 '20 at 14:26
  • update: found out that OpenShift MongoDB images only go up to `v3.6.3`. if anyone can provide a solid, best-practise, solution to achieve the desired behaviour, regardless of MongoDB version constraints, that would be great - i still need to know how it can be achieved and will sort out the MongoDB versions later. – user1063287 Aug 02 '20 at 07:25

2 Answers2

1

Here an example with mongo 3.5

const mongo = require('mongodb')

;(async function (params) {
  const client = await mongo.connect('mongodb://localhost:27017')

  const coll = client.db('test').collection('test')

  const from0to99 = Array(100).fill('0').map((_, i) => String(i))
  const from5To28 = Array(24).fill('0').map((_, i) => String(i + 5))

  const insert = { statements: from0to99.map(_ => ({ id: _ })) }
  await coll.insertOne(insert)

  const all100ElementsRead = await coll.findOneAndUpdate(
    { _id: insert._id },
    {
      $pull: {
        statements: {
          id: { $in: from5To28 }
        }
      }
    },
    { returnOriginal: true }
  )
  /**
   * It shows the object with the desired _id BEFORE doing the $pull
   * You can process all the old elements as you wish
   */
  console.log(all100ElementsRead.value.statements)

  // I use the object read from the database to push back
  // since I know the $in condition, I must filter the array returned
  const pushBack = all100ElementsRead.value.statements.filter(_ => from5To28.includes(_.id))

  // push back the 5-28 range at position 72
  const pushed = await coll.findOneAndUpdate(
    { _id: insert._id },
    {
      $push: {
        statements: {
          $each: pushBack,
          $position: 72 // 0-indexed
        }
      }
    },
    { returnOriginal: false }
  )
  console.log(pushed.value.statements) // show all the 100 elements

  client.close()
})()

This old issue helped

Manuel Spigolon
  • 11,003
  • 5
  • 50
  • 73
  • if i am reading this correctly, 1) it uses `$in` to do the initial `$pull` - as mentioned in OP, this doesn't preserve the order of the elements provided in `$in` so it is no good for pulling the elements from the array and preserving their order so that they can be pushed back into the array at a new position 2) the `$push` seems to be given new values (not the values originally pulled), so it doesn't seem to fulfil the desired behaviour - ie pull a range of values from an array and re-insert them at a different index. apologies if i am misreading it. – user1063287 Aug 04 '20 at 16:07
  • I have simplified the code in order to be more readable. 1) since you read the value before change it, the order you get will be the same 2) so don't you want to push the elements ad the and or beginning but `id: five` at position 3 and `id: six` at position 1 for example? – Manuel Spigolon Aug 04 '20 at 16:33
  • you are using $in to pull objects from an array, the order you define using $in is not necessarily the order they will come back in. therefore, if you say ‘x equals the result of remove 1,2,3’ , x may equal 3,1,2. and when you insert x back into the array all the order is wrong. i don’t understand what you mean by ‘since you read the value before change it, the order you get will be the same’. using your example, how do you pull ‘two’ and ‘three’, and put them back in the array after ‘four’? (taking into account, real scenario could be like ‘move objects 5 - 28 to position 72) – user1063287 Aug 04 '20 at 17:10
  • Thanks to `returnOriginal: true` you will get the original data, give me a bit to update the response – Manuel Spigolon Aug 05 '20 at 06:32
  • @user1063287 updated. Notice that you may evaluate `$position` as you want – Manuel Spigolon Aug 05 '20 at 06:54
-1

if you want "desired behavior" when mutating arrays , you add these to checklist:

  • array.length atleast==7 if you want to add ,splice at 6
  • creates a new array if u use concat
  • mutates orignal if used array.push or splice or a[a.length]='apple'

USE slice() to select between incex1 to index2.

or run a native for loop to select few elements of array or

apply a array.filter() finction.

once you select your elements which needed to be manupulated you mentioned you want to add it to end. so this is the method below. about adding elements at end:

CONCAT EXAMPLE

const original = ['']; //const does not mean its immutable just that it cant be reassigned
let newArray;

newArray = original.concat('');
newArray = [...original, ''];

// Result
newArray; // ['', '']
original; // ['']

SPLICE EXAMPLE:

const zoo = ['', ''];

zoo.splice(
  zoo.length, // We want add at the END of our array
  0, // We do NOT want to remove any item
  '', '', '', // These are the items we want to add
);

console.log(zoo); // ['', '', '', '', '']
nikhil swami
  • 2,360
  • 5
  • 15
  • im talking about the nodejs part assuming you first make a query then execute it to fetch data from mongoDB. if you want a mongo specific answer ill learn it and update the answer later... – nikhil swami Aug 03 '20 at 17:28