Javascript Array - Remove duplicates based on another property

Question

What is the most efficient method to dedupe a complex array based on the value of another property? I have found many examples that will dedupe an array or complex array but not this specific use case.

I am trying to find records with unique values in column 3 (state) with the highest number in column 2 (license)

var arrayWithDuplicates = [
    ["Boat", 1, "NV"],
    ["Car", 7, "CA"],
    ["Boat", 3, "NV"],
    ["Boat", 4, "CA"],
    ["Car", 5, "OR"],
    ["Boat", 6, "CA"],
];

Desired outcome

var outputArray = [
    ["Car", 7, "CA"],
    ["Boat", 3, "NV"],
    ["Car", 5, "OR"]
];

This works but not sure if with large datasets

var arrayWithDuplicates = [
  ["Boat", 1, "NV"],
  ["Car", 7, "CA"],
  ["Boat", 3, "NV"],
  ["Boat", 4, "CA"],
  ["Car", 5, "OR"],
  ["Boat", 6, "CA"],
];
let arr= arrayWithDuplicates;
let unique = []
for (let i = 0; i < arr.length; i++) {
  let found = false;
  for (let j = 0; j < unique.length; j++) {
    if (arr[i][2] === unique[j][2]) {
      found = true;
      if (arr[i][1] > unique[j][1]) {
        unique[j] = arr[i];
      }
      break;
    }
  }
  if (!found) {
    unique.push(arr[i])
  }
}
console.log(unique);

[["Boat", 3, "NV"], ["Boat", 7, "CA"], ["Car", 5, "OR"]]

You can see the performance of the proposed solutions: https://jsbench.me/eskxxcwnhn/1

Does this answer your question? [Find Max value of each x value in JavaScript multidimensional array](https://stackoverflow.com/questions/28491628/find-max-value-of-each-x-value-in-javascript-multidimensional-array) — pilchard, Jan 02 '22 at 13:31
Your original question had no visible effort and that will be voted down here — mplungjan, Jan 02 '22 at 15:37
@3limin4t0r - your solution is much faster for a larger dataset. thanks. https://jsbench.me/eskxxcwnhn/2 — Barry MSIH, Jan 02 '22 at 17:09

score 1 · Accepted Answer · answered Jan 08 '22 at 12:15

One way to solve your problem would be to use a Map instance to hold the most relevant value. Then replace it if you encounter another. Then, when you are done iterating, you take the values that are present in the Map instance.

const arrayWithDuplicates = [
  ["Boat", 1, "NV"],
  ["Car", 7, "CA"],
  ["Boat", 3, "NV"],
  ["Boat", 4, "CA"],
  ["Car", 5, "OR"],
  ["Boat", 6, "CA"],
];

const lookup = new Map();
for (const record of arrayWithDuplicates) {
  const key = record[2];
  
  if (!lookup.has(key)) {
    lookup.set(key, record);
    continue;
  }
  
  const other = lookup.get(key);
  if (record[1] > other[1]) {
    // Iteration order is based on insertion order. By removing the
    // current value first, the new value will be placed at the end.
    // If you don't care about the order, deletion can be omitted.
    lookup.delete(key);
    lookup.set(key, record);
  }
}

const result = Array.from(lookup.values());
console.log(result);

Note that the following code sequence can be a fairly heavy operation:

lookup.delete(key);
lookup.set(key, record);

Due to the fact that it rearranges the iteration order of the Map contents. This is only done get the result in the order you are looking for. If the order of the resulting items is irrelevant, then you should remove the lookup.delete(key) call to improve the execution speed.

Although using Map instances might produce some execution overhead for small collections. The improved lookup speed really shines when collections get larger.

mplungjan · Answer 2 · 2022-01-02T13:53:44.490

Using Sort and Reduce

const arrayWithDuplicates = [ ["Boat", 1, "NV"], ["Car", 7, "CA"], ["Boat", 3, "NV"], ["Boat", 4, "CA"], ["Car", 5, "OR"], ["Boat", 6, "CA"], ]; 

let deduped = [...arrayWithDuplicates]; // take a copy

deduped.sort((a, b) => { // sorts in place
  if (a[0] < b[0]) return 1; // sort on names
  if (a[0] > b[0]) return -1;
  return b[1] - a[1]; // sort on second element
})

// reduce into an object keyed on state
deduped = Object.values(  // take only the values from the object
  deduped.reduce((acc, cur) => {
    const state = cur[2];
    if (!acc[state]) acc[state] = cur;
    return acc;
},{}))
console.log(deduped)

Javascript Array - Remove duplicates based on another property

2 Answers2