node: removing duplicate objects from array

Question

I have an array of objects and I would like to remove duplicate items. The following discusses the topic however the ES6 solutions do not appear to work with objects: Unique values in an array

var myArray=[{fname:"Bob", sname:"Brown"},{fname:"Jill",sname:"White"},{fname:"Bob", sname:"Brown"}];
var unique = myArray.filter((v, i, a) => a.indexOf(v) === i);
/*
[ { fname: 'Bob', sname: 'Brown' },
  { fname: 'Jill', sname: 'White' },
  { fname: 'Bob', sname: 'Brown' } ]
*/
let unique = [...new Set(myArray)];
/*
[ { fname: 'Bob', sname: 'Brown' },
  { fname: 'Jill', sname: 'White' },
  { fname: 'Bob', sname: 'Brown' } ]
*/

Mech45 · Accepted Answer · 2017-01-14T08:05:32.887

5

Use _.uniqBy or _.uniqWith from lodash ;)

const _ = require('lodash');

const myArray=[{fname:"Bob", sname:"Brown"},{fname:"Jill",sname:"White"},{fname:"Bob", sname:"Brown"}];

console.log(_.uniqWith(myArray, _.isEqual));

edited Jan 14 '17 at 08:05

answered Jan 13 '17 at 10:40

Mech45

321
1
9

Code works thank you. Having trouble understanding the difference between uniq.js, uniqBy.js & uniqWith.js – Dercni Jan 14 '17 at 02:06

rsp · Answer 2 · 2017-01-13T10:46:51.407

With lodash you can use _.uniqBy or _.uniqWith - see the answer by Steven Leclerc.

See: https://lodash.com/ for details.

If you're serious about writing code then you shouldn't reinvent the wheel for simple cases like this.

If you want to know how to implement it properly, see the source code here:

Keep in mind that when you use JSON.stringify then first of all you must put it inside of a try/catch block because your program will crash for some data (or you should use tryjson - disclaimer, I'm the author of that module) and that you will get false positives of objects that are different but have the same JSON serialization.

@StevenLeclerc thanks. I updated my answer and also referred to yours. — rsp, Jan 13 '17 at 10:51

score 0 · Answer 3 · answered Jan 13 '17 at 10:39

A solution would be to use the same solution you referred to after a small modification; as using Objects as keys is not possible, we can use them as JSON which is string:

Array.prototype.getUnique = function() {
    var u = {}, a = [];
    for(var i = 0, l = this.length; i < l; ++i) {
        var index = JSON.stringify(this[i]);
        if(u.hasOwnProperty(index)) {
            continue;
        }
        a.push(this[i]);
        u[index] = 1;
    }
    return a;
}

Hope this helps :)

score 0 · Answer 4 · answered Jan 13 '17 at 10:47

0

Without lodash:

function removeDuplicates(myArr, prop) {
  return myArr.filter((element, pos, arr) => {
    return arr.map(mapObj => mapObj[prop]).indexOf(element[prop]) === pos;
  });
}

answered Jan 13 '17 at 10:47

jesusbv - user3085938

174
1
4

score -1 · Answer 5 · answered Jan 13 '17 at 10:58

Here first and last element does not represent same object, therefore it is not removed from set

var myArray=[{fname:"Bob", sname:"Brown"},{fname:"Jill",sname:"White"},{fname:"Bob", sname:"Brown"}];
var set1 = new Set(myArray);
console.log([...set1]);
/*
[ { fname: 'Bob', sname: 'Brown' },
  { fname: 'Jill', sname: 'White' },
  { fname: 'Bob', sname: 'Brown' } ]
*/

Here only one object is added to set, as both are same.

var set2 = new Set();
var obj1 = {fname:"Bob", sname:"Brown"};
set2.add(obj1);
set2.add(obj1);
console.log([...set2]);
/*
[ { fname: 'Bob', sname: 'Brown' }]
*/

You have to compare manually both object content, and removed any duplicates.

node: removing duplicate objects from array

5 Answers5