Delete duplicated elements in array of objects Javascript

Question

Basically I want to know how to clean an array of objects in JavaScript.

I have seen some samples that show how to do it depending on the value of one element and I need it to be generic. It means that it doesn't depend on the child's names.

Example of the way i don't want

What I need is a function that can work independently of the object structure.

For example, it can work with this:

object = [
    {value: 'value', configuration: 'configuration'},
    {value: 'value2', configuration: 'configuration2'},
    {value: 'value', configuration: 'configuration'},
]

//returning:

object = {
    {value: 'value', configuration: 'configuration'},
    {value: 'value2', configuration: 'configuration2'}
}

And it can also work with:

object = [
    {name: 'name', property: 'property'},
    {name: 'name2', property: 'property2'},
    {name: 'name', property: 'property'},
]

//returning:

object = [
    {name: 'name', property: 'property'},
    {name: 'name2', property: 'property2'}
]

there are multiple ways to delete duplicates, which vary in time & space complexity — ControlAltDel, Dec 15 '17 at 18:44
Objects can contain nested objects of various types (Maps, Sets, dates, your own class instances, functions, regexes, HTML DOM nodes, ...), and have cyclic references (a child having the parent object as property value; or a cross reference to another object in the parent object/array). What you consider duplicate becomes quite ambiguous in those circumstances. Please provide clear boundaries for your question. Also your first paragraph speaks of arrays of objects, but the code examples use invalid syntax. — trincot, Dec 15 '17 at 18:47
Will your objects only contain Strings, Numbers, Arrays, Objects, Bools? No functions and dates? — Cory Danielson, Dec 15 '17 at 18:48
FYI your code does not contain valid syntax. Are those `object = {...}` supposed to be arrays of objects? — Cory Danielson, Dec 15 '17 at 18:54
@CoryDanielson It will contain objects... Thanks for the correction. — Hely Saul Oberto, Dec 15 '17 at 18:58

score 2 · Accepted Answer · answered Dec 15 '17 at 19:01

Deduplicating an array is straightforward using the filter/findIndex trick, the only problem is that we need an equality function to use to compare elements. For objects, a simple === won't work.

Here is a simple example using a basic shallow equals comparator. It only checks that two objects have the same values for the same keys, which will work for the objects in your example (but it wouldn't work with nested objects). Depending on your use case that may not be sufficient. Another good option would be lodash's _.isEqual method, or a function to compare their JSON strings. The important thing is that you define a function to compare two objects that meets your requirements.

var array1 = [
    {value: 'value', configuration: 'configuration'},
    {value: 'value2', configuration: 'configuration2'},
    {value: 'value', configuration: 'configuration'},
];

var array2 = [
    {name: 'name', property: 'property'},
    {name: 'name2', property: 'property2'},
    {name: 'name', property: 'property'},
];

// You need to define an equality function to use with the deduplicator. Heres a basic one.
function isKVEqual(obj1, obj2) {
  // Get the keys of these objects, make sure they have the same number of keys.
  const o1keys = Object.keys(obj1);
  const o2keys = Object.keys(obj2);
  if (o1keys.length !== o2keys.length) return false;
  
  // Check that the value of each key is the same in each object.
  for (const key of o1keys) {
    if (obj2[key] !== obj1[key]) return false;
  }
  
  return true;
}

// Deduplicate:
// Make sure to replace the isKVEqual call with whatever custom comparator you want to use.
const dedupedArray1 = array1.filter((item, index, list) => list.findIndex(x => isKVEqual(x, item)) === index);
const dedupedArray2 = array2.filter((item, index, list) => list.findIndex(x => isKVEqual(x, item)) === index);

console.log(dedupedArray1)
console.log(dedupedArray2)

score 1 · Answer 2 · answered Dec 15 '17 at 18:54

If you're willing to use an external library, I would suggest using lodash to check if the 2 items are duplicates, and filtering that way:

let object = [
    {name: 'name', property: 'property'},
    {name: 'name2', property: 'property2'},
    {name: 'name', property: 'property'},
]

function removeDuplicates(arr) {
  return arr.filter((elem, idx) => {
    return !arr.slice(idx + 1).find((other) => _.isEqual(elem, other));
  });
}

console.log(removeDuplicates(object));

<script src="https://cdn.jsdelivr.net/npm/lodash@4.17.4/lodash.min.js"></script>

Ciro Spaciari · Answer 3 · 2017-12-15T21:50:21.423

You can use filter and json for generic check

var list = [
    {value: 'value', configuration: 'configuration'},
    {value: 'value2', configuration: 'configuration2'},
    {value: 'value', configuration: 'configuration'},
];
//filter element
list = list.filter(function(value, index){
  var exists = false;
  //verify if exists the same element with other index before
  for(var i in list){
    //generic verify the json
    if(JSON.stringify(list[i]) ==         JSON.stringify(list[index]) && i < index){            exists = true; 
       break;
    }
  }  
  //if dont exists dont filter
  return !exists;
});

console.log(list);

you can use a custom compare instead JSON.stringfy

function isEqual(a, b){

for(var i in a)
       if(a[i] != b[i])
          return false;
  for(var i in b)
       if(b[i] != a[i])
          return false;
  return true;
}

because JSON.stringfy generate different strings for

{value: 'value', configuration: 'configuration'},

and

{ configuration: 'configuration', value: 'value'},

Delete duplicated elements in array of objects Javascript

3 Answers3

Linked