how to remove duplicate objects from an Array of objects but with two unique keys

Question

So I have an array of objects and I am trying to remove the duplicates. For example, lets say my array of object contains this data:

[{TERM_CD: "220181", CRS_SUBJ_CD: "LALS", CRS_SUBJ_DESC: "Latin American &Latino Studies", CRS_NBR: "127", CRS_TITLE: "Latin American Music", …}, 
{TERM_CD: "220101", CRS_SUBJ_CD: "MUS", CRS_SUBJ_DESC: "Music", CRS_NBR: "127", CRS_TITLE: "Latin American Music", …}, etc...]

And this is the removeDup function that I am using to remove duplicates:

removeDup(data, key) {
    return [
      ...new Map(
        data.map(x=>[key(x), x])
      ).values(),
    ];
  }

For example if I call the remove dup function like this then one of the object will be removed since they both have same crs_title.

const noDupArr = this.removeDup(printArr, x => x.CRS_TITLE);

So my goal is to try to remove duplicate on 2 keys CRS_SUBJ_CD and CRS_NBR but I am not able to figure out how to proceed doing that. I tried adding another key parameter in the removeDup function but wasn't successful. Any ideas on fixing this issue. Thank you in advance!

_my goal is to try to remove duplicate on 2 keys CRS_SUBJ_CD and CRS_NBR_. If your `removeDup` function is working, then why not call it twice with the other keys you want to remove — smac89, Jun 26 '20 at 18:22
Calling it twice isn't really solving the issue because it removes certain objects that shouldn't be removed. — Shalin Patel, Jun 26 '20 at 18:24
You can think of using Map , where title is key and full obj is value. If it is already present , dont add else add. — Harmandeep Singh Kalsi, Jun 26 '20 at 18:25
https://stackoverflow.com/questions/50729508/removing-duplicate-objects-from-an-array/50729724#50729724 — Henrique Viana, Jun 26 '20 at 18:35
@Henrique Viana that works! Didn't get to look at the question before. Thanks! — Shalin Patel, Jun 26 '20 at 18:41

score 1 · Answer 1 · answered Jun 26 '20 at 18:39

You can maintain an array of keys for filtering and then make use of Map to get unique values. Here is an example with duplicate data:

var data=[{TERM_CD: "220181", CRS_SUBJ_CD: "LALS", CRS_SUBJ_DESC: "Latin American &Latino Studies", CRS_NBR: "127", CRS_TITLE: "Latin American Music"},
{TERM_CD: "220101", CRS_SUBJ_CD: "MUS", CRS_SUBJ_DESC: "Music", CRS_NBR: "127", CRS_TITLE: "Latin American Music"},{TERM_CD: "220101", CRS_SUBJ_CD: "MUS", CRS_SUBJ_DESC: "Music", CRS_NBR: "127", CRS_TITLE: "Latin American Music"}];

var keys = ['CRS_SUBJ_CD',  'CRS_NBR'];

var result = [...new Map(data.map(p=>[keys.map(k=>p[k]).join('|'),p])).values()];

console.log(result);

I hope this helps. Thanks!

False positives: `{CRS_SUBJ_CD:"foo", CRS_NBR:"bar"}` and `{CRS_SUBJ_CD:"foo|bar", CRS_NBR:""}` OR `{CRS_SUBJ_CD:"foo ", CRS_NBR:"|bar"}` and `{CRS_SUBJ_CD:"foo |", CRS_NBR:"bar"}` — smac89, Jun 26 '20 at 18:42

how to remove duplicate objects from an Array of objects but with two unique keys

1 Answers1