find duplicates in array per document in mongodb

Question

Let's say that I have some document with this structure:

   _id: ObjectId('444455'),
   name: 'test',
   email: 'email,
   points: {
      spendable: 23,
      history: [
          {
             comment: 'Points earned by transaction #1234',
             points: 1
          },
          {
             comment: 'Points earned by transaction #456',
             points: 3
          },
          {
             comment: 'Points earned by transaction #456',
             points: 3
          }
      ]
   }
}

Now I have a problem that some documents contains duplicates objects in the points.history array.

Is there a way to easily find all those duplicates by a query?

I already tried this query: Find duplicate records in MongoDB but that shows the total count of every duplicated line in all documents. I need a overview of the duplicates per document like this:

{
    _id: ObjectId('444455') //_id of the document not of the array item itself
    duplicates: [
       {
        comment: 'Points earned by transaction #456
       }
    ]
}, {
    _id: ObjectId('444456') //_id of the document not of the array item itself
     duplicates: [
         {
            comment: 'Points earned by transaction #66234
         },
         {
            comment: 'Points earned by transaction #7989
         }
     ]
}

How can I achieve that?

Can your sample document produce the expected output ? if Not then pls update — Ashh, Jan 29 '20 at 11:50
https://stackoverflow.com/questions/39399149/removing-duplicate-array-values-from-mongodb refer this — vicky, Jan 29 '20 at 11:53

score 2 · Accepted Answer · answered Jan 29 '20 at 11:57

2

Try below aggregate pipeline

collectionName.aggregate([
  {
    $unwind: "$points.history"
  },
  {
    $group: {
      _id: {
        id: "$_id",
        comment: "$points.history.comment",
        points: "$points.history.points"
      },
      sum: {
        $sum: 1
      },

    }
  },
  {
    $match: {
      sum: {
        $gt: 1
      }
    }
  },
  {
    $project: {
      _id: "$_id._id",
      duplicates: {
        comment: "$_id.comment"
      }
    }
  }
])

answered Jan 29 '20 at 11:57

sushant mehta

1,244
1
7
13

Nice! that's looking good. Any idea's how to add the _id of the document itself in the $project? – NVO Jan 29 '20 at 12:19
I have updated my question for better understanding – NVO Jan 29 '20 at 12:28
_id in last project will return _id of the document itself not _id of nested array comment – sushant mehta Jan 29 '20 at 12:37
Sorry! my bad! I didn't check that properly. Your answer is working fine! Thanks! – NVO Jan 29 '20 at 12:47

find duplicates in array per document in mongodb

1 Answers1