Finding nested duplicate arrays in JavaScript. (Nested Array uniq in lodash/underscore)

Question

I am trying to determine if an array of JavaScript arrays contains duplicates. Is this possible? I am first trying to see if I can strip the duplicates out and then do an equality check but I cannot get past the first part. Here is what underscore returns:

var arr1 = [[1,2], [2,3], [1,2]];
var arr2 = _.uniq(arr1);

var arraysAreEqual = _.isEqual(arr1, arr2);

console.log(arraysAreEqual, arr1, arr2);
// true

Jsbin: http://jsbin.com/vogumo/1/edit?js,console

Anyone know of a way to determine if the array contains duplicate arrays?

When you say 'duplicate', do you mean the arrays are the same object, or that the arrays have the same length and order of items, or have the same length but not order of items? — Xotic750, Jan 09 '15 at 01:19

score 2 · Answer 1 · answered Jan 09 '15 at 01:15

2

It's a little sloppy, but (possible)

var arr2 = _.uniq(arr1, function(item) {
    return JSON.stringify(item);
});

will give you a correct result

answered Jan 09 '15 at 01:15

Mark Dickson Jr.

588
4
10

In this very simple example then perhaps and depending on the OP's definition. – Xotic750 Jan 09 '15 at 11:35

score 2 · Answer 2 · edited May 23 '17 at 12:19

From the underscore.js documentation:

uniq _.uniq(array, [isSorted], [iteratee]) Alias: unique
Produces a duplicate-free version of the array, using === to test object equality. If you know in advance that the array is sorted, passing true for isSorted will run a much faster algorithm. If you want to compute unique items based on a transformation, pass an iteratee function.

But arrays can't be strictly compared in JavaScript.

Therefore, you can use a transformation function to enable comparison with uniq. For example:

console.log([1,2] === [1,2]) // false, can't strict compare arrays
console.log([1,2].toString()) // "1,2" - string representation
console.log([1,2].toString() === [1,2].toString()) // true, strings can be compared

var valueToString = function(v) {return v.toString()}; // transform array to string
var arr1 = [[1,2], [2,3], [1,2]];
var arr2 = _.uniq(arr1, false, valueToString); // compare based on transformation
var arraysAreEqual = _.isEqual(arr1, arr2);

console.log("arraysAreEqual:", arraysAreEqual, arr1, arr2); 
// false
// [[1, 2], [2, 3], [1, 2]]
// [[1, 2], [2, 3]]

Note that transforming to string is "hacky": you would be better off comparing each value of the array, as discussed in this StackOverflow question.

By using the proposed equals implementation in that question, you would need to implement your own version of uniq that uses equals instead of ===.

The implementation of uniq in Underscore is very straight-forward - it creates a new result array and loops through the given array. If the current value is not already in result, insert it.

console.log("Using array comparison:");
arrayEquals = function (array1, array2) {
    // if any array is a falsy value, return
    if (!array1 || !array2)
        return false;

    // compare lengths - can save a lot of time 
    if (array1.length != array2.length)
        return false;

    for (var i = 0, l=array1.length; i < l; i++) {
        // Check if we have nested arrays
        if (array1[i] instanceof Array && array2[i] instanceof Array) {
            // recurse into the nested arrays
            if (!arrayEquals(array1[i],array2[i]))
                return false;       
        }           
        else if (array1[i] !== array2[i]) { 
            return false;   
        }        
    }       
    return true;
};

_.uniqArrays = function(array) {
  if (array == null) return [];
  var result = [];
  for (var i = 0, length = array.length; i < length; i++) {
    var value = array[i];
    var arrayEqualsToValue = arrayEquals.bind(this, value); // arrayEquals with first argument set to value
    var existing = _.find(result, arrayEqualsToValue); // did we already find this?
    if (!existing) {
      result.push(value);
    }
  }
  return result;
};

var arr3 = _.uniqArrays(arr1);
arraysAreEqual = _.isEqual(arr1, arr3);
console.log("arraysAreEqual:", arraysAreEqual, arr1, arr3); // false

I made a jsbin with all the code, if you want to play around.

Arrays can be strictly compared, `var a=[1,2], b=[1,2];` now `a===a` is `true`, they are the same array. But `a===b` is `false`, they are not the same array but they are both arrays of the same length and same order of item values. To test their equality for item values then you should check their length and then item by item, same order or not. — Xotic750, Jan 09 '15 at 11:30
@Xotic750, when you compare `a === a` you are not comparing the array to itself, you are comparing the **reference** to the array to itself, which obviously must be true. Actually, `a === a` will always be true ([jsbin](http://jsbin.com/xavoba/2/edit?js,console)). You are correct that the best way to check for equality is to check for each item, though, and I will link to an answer that discusses that. — Guilherme Rodrigues, Jan 09 '15 at 11:45
Then they are being strictly compared as it is the same object, and I did go on to explain but perhaps it was not clear enough as just a comment. I also questioned the OP about his meaning. And `a===a` is not always `true`, try `var a=NaN, b=NaN`. Perhaps your statement should have read something like `You can not strictly compare (i.e. === or even ==) 2 different array object for their content` — Xotic750, Jan 09 '15 at 11:53
And even more than the simple question that I asked the OP you may even need to consider if the following are counted as being equal `var a=[1,2], b={0:1,1:2,length:2}`, as just another example — Xotic750, Jan 09 '15 at 12:08
I have improved my question to reflect that transforming to string is not the ideal way to go about it. Anyway, we must agree that it's way simpler :) — Guilherme Rodrigues, Jan 09 '15 at 12:14
There is actually a well agreed upon [equivalence assertion test](http://wiki.commonjs.org/wiki/Unit_Testing/1.0) (aka `deepEqual`), and that may actually suit the OP's need, but who knows. — Xotic750, Jan 09 '15 at 12:24

score 2 · Answer 3 · answered Sep 04 '16 at 15:49

Try This:

var numArray = [1, 7, 3, 0, 9, 7, 8, 6, 2, 3];
var duplicates = [];
var sortednumArray = numArray.sort();


for (var i = 0; i < sortednumArray.length; i++) {
    //console.log(sortednumArray[i]);
    if (sortednumArray[i] == sortednumArray[i + 1]) {
        duplicates.push(sortednumArray[i]);
    }
}

if (duplicates.length == 0) {
    console.log("Soted Array:");
    for(var i = 0; i < sortednumArray.length; i++) {
        console.log(sortednumArray[i]);
    }
} else {
    console.log("Duplicates:");
    for(var i = 0; i < duplicates.length; i++){
        console.log(duplicates[i]);
    }
}

Program pushes all duplicates to an array called 'duplicates' then displays it, but if none are present, it displays the sorted version of numArray

score 1 · Answer 4 · answered Mar 31 '16 at 09:18

1

In the latest lodash (4.6.1) you could do something like this:

if (_.uniqWith(arr, _.isEqual).length < arr.length) {
  // then there were duplicates
}

answered Mar 31 '16 at 09:18

Fergie

5,933
7
38
42

Finding nested duplicate arrays in JavaScript. (Nested Array uniq in lodash/underscore)

4 Answers4

Linked