Remove Duplicate elements from Array - Javascript (No JQuery & ECMAScript)

Question

Case: We have 'n' number of arrays stored in an array (Array of Arrays). Now that each child array in this parent array can have elements that may or may not be present in other child arrays. Output - I need to create an array which has the all the elements present in all the child arrays excluding the duplicates.

I do not want to concatenate all the arrays into a single array and use unique method to filter out. I need to create unique array then and there during iteration.

Ex:

var a[] = [1,2,3,4,5];
var b[] = [1,2,7,8];
var c[] = [1,2,3,4,5,6,7,8];
var d[] = [9,10,11,12];
var arr[] = [a,b,c,d]
Output must be [1,2,3,4,5,6,7,8,9,10,11,12]

P.S: I can concat the arrays and use jquery unique function to resolve this, but i need a solution in javascript alone. Thanks

Answer 1

You can use array#reduce to flatten your array and then use Set to get distinct values and use array#from to get back array from Set.

var a = [1,2,3,4,5];
var b = [1,2,7,8];
var c = [1,2,3,4,5,6,7,8];
var d = [9,10,11,12];
var arr = [a,b,c,d]

var result = Array.from(new Set(arr.reduce((r,a) => r.concat(a))));
console.log(result);

Answer 2

Try using .filter when adding each array to the final one, filtering out the duplicates:

a.filter(function(item) {
    return !finalArray.contains(item));
});

Answer 3

Answer using Sets:

var a = [1,2,3,4,5];
var b = [1,2,7,8];
var c = [1,2,3,4,5,6,7,8];
var d = [9,10,11,12];

var concat = a.concat(b).concat(c).concat(d);
var union = new Set(concat);
//console.log(union);

ES6 Answer:

let a = new Set([1,2,3,4,5]);
let b = new Set([1,2,7,8]);
let c = new Set([1,2,3,4,5,6,7,8]);
let d = new Set([9,10,11,12]);

let arr = new Set([...a,...b,...c,...d]);
//Result in arr.

Whats going on???

From https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set:

The Set object lets you store unique values of any type, whether primitive values or object references.

So when we initialise Sets passing arrays to the constructor we basically ensure that there are no duplicate values.

Then in the last line, we concat all the Sets we initialised prior into a final set.

The ... notation converts the Set into an array, and when we pass the 4 arrays to the constructor of the Set they get concatenated and a Set of their unique values is created.

Answer 4

Here is a functional alternative written in ES5.

var flatten = function(list) {
  return list.reduce(function(acc, next) {
    return acc.concat(Array.isArray(next) ? flatten(next) : next);
  }, []);
};

var unique = function(list) {
  return list.filter(function(element, index) {
    return list.indexOf(element) === index;
  })
}


var a = [1,2,3,4,5];
var b = [1,2,7,8];
var c = [1,2,3,4,5,6,7,8];
var d = [9,10,11,12];
var arr = [a,b,c,d];

var result = unique(flatten(arr));

console.log(result);

If you support ES6, arrow function can make that code even shorter.

Answer 5

Here is a solution that uses a plain object for resolving duplicates, and only uses basic ES3 JavaScript. Runs in IE 5.5 and higher, and with O(n) time complexity.

function uniques(arr) {
    var obj = {}, result = [];
    for (var i = 0; i < arr.length; i++) {
        obj[arr[i]] = true;
    }
    for (var prop in obj) {
        if (obj.hasOwnProperty(prop)) result.push(+prop);
    }
    return result;
}

// Example use
var a = [1,2,3,4,5], 
    b = [1,2,7,8],
    c = [1,2,3,4,5,6,7,8],
    d = [9,10,11,12];

var result = uniques(a.concat(b, c, d));

console.log('Result: ' + result);

As an object can only have a unique set of properties (no duplicates), the use of all array values as properties in an object will give you an object with a property for each unique value. This happens in the first loop. NB: the value given to those properties is not relevant; I have used true.

Then the result is just the conversion of those properties back to array values. This happens in the second loop.

Answer 6

var a = [1,2,3,4,5];
var b = [1,2,7,8];
var c = [1,2,3,4,5,6,7,8];
var d = [9,10,11,12];
var result = a.concat(b,c,d);
function remvDup(result){
    var tmp = [];
    for(var i = 0; i < result.length; i++){
        if(tmp.indexOf(result[i]) == -1){
        tmp.push(result[i]);
        }
    }
    return tmp;
}
console.log(remvDup(result));

Answer 7

Becuase the OP mentioned that he cannot use 'Set' as it is not supported on the targeted browsers, I would recommand using the 'union' function from the lodash library.

See union's documentation here