Remove duplicate elements of an array, in an object of arrays, dynamically

Question

I have checked other solutions but none fit the criterion of my problem

This solution does not have the ability to dynamically check each node

Problem summarized

I wish to create an algorithm that is able to check an object that has nodes of different data types, for duplicated objects in nodes that are specifically of the datatype array.

I have the following dataset:

task = {
  content: "lorem....",
  customer: [
    { id: 1, name: "hello" },
    { id: 2, name: "sup" },
  ],
  end: "2020-08-13 10:09:48",
  project: [{ id: 1 }, { id: 1 }, { id: 2 }],
  vendor: [{ id: 2 }, { id: 2 }, { id: 3 }],
};

I wish to be able to dynamically check which of the objects (or nodes? and the algo has to recognize that it is an array) has duplicates, and reduce them to be in this form:

task = {
  content: "lorem....",
  customer: [
    { id: 1, name: "hello" },
    { id: 2, name: "sup" },
  ],
  end: "2020-08-13 10:09:48",
  project: [{ id: 1 }, { id: 2 }],
  vendor: [{ id: 2 }, { id: 3 }],
};

EDIT

The algorithm needs to be able to handle a dynamic number of nodes (example 1), however , the duplicates will only happen 1 level down (Thanks for pointing out).

example 1 (there is 1 less node here ) :

task = {
  content: "lorem....",
  customer: [
    { id: 1, name: "hello" },
    { id: 2, name: "sup" },
  ],
  end: "2020-08-13 10:09:48",
  project: [{ id: 1 }, { id: 2 }],
};

"algo has to recognize that it is an array" - see [Array.isArray](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/isArray) — Jan Stránský, Aug 13 '20 at 14:25
Hello , thanks for the comment , im aware of such a function to detect if its an array , however i havent have a clue on how to apply this to get the results i want — neowenshun, Aug 13 '20 at 14:26
"I have checked other solutions but non fits the criterion of my problem" - please provide some links and why it did not work — Jan Stránský, Aug 13 '20 at 14:26
Can you show what you've tried thus far, and where you've run into problems? — Alexander Nied, Aug 13 '20 at 14:27
@JanStránský Hey i have made an edit to the post , the link is one of the links that i have visited. In general stackoverflow has many questions and answers that solves 1 part of my problem which is to filter out duplicates in an array , however i need the algo to be able to dynamically check each node as the dataset may or may not change the number of nodes — neowenshun, Aug 13 '20 at 14:29
@AlexanderNied Hi Alexander , Im only able to remove duplicates from a single array , however as mentioned above , that only solves one part of my problem , i need to be able to check a dynamic number of nodes — neowenshun, Aug 13 '20 at 14:30
There are many unknowns in the "assignment". E.g. are the arrays only in the "first level" of the object, or there can be nested (possibly complex) object-array structures? — Jan Stránský, Aug 13 '20 at 14:33
@JanStránský Thanks for pointing the mistake out , i have made an edit to the post — neowenshun, Aug 13 '20 at 14:37

score 1 · Accepted Answer · answered Aug 13 '20 at 14:31

1

Here is my proposed solution to remove duplicate elements from any array in the task object:

const uniq = array => {
  const map = {};
  const result = [];
  for (let i = 0; i < array.length; i++) {
    // since elements can be objects, need to do a deep comparison.
    const element = JSON.stringify(array[i]);
    if (map[element] === undefined) {
      map[element] = true;
      result.push(array[i]);
    }
  }
  return result;
}

const task = {
  content: "lorem....",
  customer: [
    { id: 1, name: "hello" },
    { id: 2, name: "sup" },
  ],
  end: "2020-08-13 10:09:48",
  project: [{ id: 1 }, { id: 1 }, { id: 2 }],
  vendor: [{ id: 2 }, { id: 2 }, { id: 3 }],
};

for (const key in task) {
  if (Array.isArray(task[key])) {
    task[key] = uniq(task[key])
  }
}

console.log('deduped:', task);

answered Aug 13 '20 at 14:31

Ethan Lipkind

1,136
5
7

Thanks this works perfectly for me , appreciate the help , im having a better understanding of JS everyday – neowenshun Aug 13 '20 at 14:41
1

`for (let i = 0; i < array.length; i++) { ... array[i] ... }` **I** prefer `of` solution: `for (let e of array) { ... e ... }` (not sure if it is worth editing the answer) – Jan Stránský Aug 13 '20 at 14:43
using foreach would probably be more idiomatic, but I'm just gonna leave it since the vanilla for loop demonstrates what's going on under the hood. – Ethan Lipkind Aug 13 '20 at 14:54
[When using a `for...in` loop, you should probably be doing some checks for own-properties, or consider using a different looping method that will exclude inherited properties](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for...in#Iterating_over_own_properties_only) – Alexander Nied Aug 13 '20 at 15:18

Remove duplicate elements of an array, in an object of arrays, dynamically

1 Answers1