Delete duplicate elements from an array

Question

For example, I have an array like this;

var arr = [1, 2, 2, 3, 4, 5, 5, 5, 6, 7, 7, 8, 9, 10, 10]

My purpose is to discard repeating elements from array and get final array like this;

var arr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

How can this be achieved in JavaScript?

NOTE: array is not sorted, values can be arbitrary order.

You could loop over the array and copy all elements to a map. — Devolus, May 25 '13 at 08:30
@Devolus that's only valid for strings and numbers, you cannot use object or array as index key. — Umur Kontacı, May 25 '13 at 08:31
It's a solution but, is also brute force solution. Is there smarter (efficient) way of this? — Mehmet Ince, May 25 '13 at 08:31
Sort the array, iterate, push element to new array if not the same as last. — DarthJDG, May 25 '13 at 08:34
@UmurKontacı, if javascript is similar to java, then you have corresponding classes to primitives, like Boolean for boolean, Integer for int and so on. So it can be done this way. — Devolus, May 25 '13 at 08:40

score 265 · Accepted Answer · edited Nov 29 '17 at 11:56

265

It's easier using Array.filter:

var unique = arr.filter(function(elem, index, self) {
    return index === self.indexOf(elem);
})

edited Nov 29 '17 at 11:56

antpaw

15,444
11
59
88

answered May 25 '13 at 08:47

Niccolò Campolungo

11,824
4
32
39

9

how is this efficient ? Care to explain worst case run time depends on how indexOf works if it runs in linear time, then the worst case time is O(n^2) – abkds Jan 12 '16 at 15:52
@AbKDs please elaborate more on this – Eduardo in Norway Jul 31 '16 at 17:19
1

@MehmetInce it is not "efficient" at all, that runs in O(n^2) – Dragos Rizescu Mar 30 '17 at 12:24
how would you also remove both items if there are any duplicates? – SuperUberDuper Apr 14 '17 at 10:18
Could someone expound on how this magic works? :-) – MadPhysicist May 09 '17 at 19:58
5

.filter() iterates over all the elements of the array and returns only those for which the callback returns true. .indexOf() returns the index of the leftmost element in the array. If there are duplicate elements, then for sure they're gonna be removed when their index is compared to the leftmost one. For more info check https://developer.mozilla.org/it/docs/Web/JavaScript/Reference/Global_Objects/Array/filter :) @MadPhysicist – Niccolò Campolungo May 09 '17 at 21:58
@SuperUberDuper same structure with the filter callback, just change the condition to one that is true when there are more than two elements in the array (it's another function, not sure if there's a default countElements or something like that by default in JS). Sorry if I didn't respond sooner, I must have missed the notification! – Niccolò Campolungo May 09 '17 at 22:01
@Niccolò This is really inefficient: O(n²), compared to O(n) if you keep track of already-seen items as in Denys' answer (although I would use an ES6 Set rather than an object). This assumes the JavaScript implementation uses a hash table, which has O(1) insert/lookup time, but even if it uses a tree, which has O(log n) operations, then the total complexity would be O(n log n), which is still much better than O(n²). That was the point AbKDs was making, which your discussion of filter just confirmed. – Arthur Tacca Dec 09 '17 at 18:34
1

This answer is still fine for very small lists though. It has the advantage of being very simple code, so less likely to have bugs in and less effort to read than code using using an auxillary store. It's less flexible though: it wouldn't work if you wanted to eliminate duplicates based on only a single field of items in the list, whereas the other solution would be easy to adapt to that. – Arthur Tacca Dec 09 '17 at 18:38
hi, how if i just want to delete only 1, for example there are 3 numbers of 5 in array, so how to make it only 2 numbers of 5 ? – M. Pancadewa Mar 27 '19 at 21:56
Clever and useful for many cases, but it's worth noting that does not work for arrays of arrays. – Vacilando Sep 11 '19 at 21:17

Denys Séguret · Answer 2 · 2019-06-17T04:58:56.020

24

As elements are yet ordered, you don't have to build a map, there's a fast solution :

var newarr = [arr[0]];
for (var i=1; i<arr.length; i++) {
   if (arr[i]!=arr[i-1]) newarr.push(arr[i]);
}

If your array weren't sorted, you would use a map :

var newarr = (function(arr){
  var m = {}, newarr = []
  for (var i=0; i<arr.length; i++) {
    var v = arr[i];
    if (!m[v]) {
      newarr.push(v);
      m[v]=true;
    }
  }
  return newarr;
})(arr);

Note that this is, by far, much faster than the accepted answer.

edited Jun 17 '19 at 04:58

answered May 25 '13 at 08:33

Denys Séguret

372,613
87
782
758

1

array is not always sorted. – Mehmet Ince May 25 '13 at 08:37
Its easier than that Making your approach a bit simpler var m = []; arr.forEach( v => m[v]=true ); var newarr = Object.keys(m); Easy, right? – pPanda_beta Aug 14 '17 at 15:06
1

This is a great answer. I would use an ES6 Set rather than an object though, so that the items in the list don't have to be strings. (Obviously that wasn't an option back when the answer was written.) – Arthur Tacca Dec 09 '17 at 19:24
It's extremely fast with unordered arrays too. Thanks! – German Feb 25 '21 at 18:48
Awesome answer! ;) – Britto Sep 05 '21 at 23:43

score 11 · Answer 3 · answered May 25 '13 at 08:37

var arr = [1,2,2,3,4,5,5,5,6,7,7,8,9,10,10];

function squash(arr){
    var tmp = [];
    for(var i = 0; i < arr.length; i++){
        if(tmp.indexOf(arr[i]) == -1){
        tmp.push(arr[i]);
        }
    }
    return tmp;
}

console.log(squash(arr));

Working Example http://jsfiddle.net/7Utn7/

Compatibility for indexOf on old browsers

score 6 · Answer 4 · answered May 25 '13 at 08:34

6

you may try like this using jquery

 var arr = [1,2,2,3,4,5,5,5,6,7,7,8,9,10,10];
    var uniqueVals = [];
    $.each(arr, function(i, el){
        if($.inArray(el, uniqueVals) === -1) uniqueVals.push(el);
    });

answered May 25 '13 at 08:34

sAnS

1,169
1
7
10

Grijesh Chauhan · Answer 5 · 2013-05-25T11:29:09.097

5

Try following from Removing duplicates from an Array(simple):

Array.prototype.removeDuplicates = function (){
  var temp=new Array();
  this.sort();
  for(i=0;i<this.length;i++){
    if(this[i]==this[i+1]) {continue}
    temp[temp.length]=this[i];
  }
  return temp;
}

Edit:

This code doesn't need sort:

Array.prototype.removeDuplicates = function (){
  var temp=new Array();
  label:for(i=0;i<this.length;i++){
        for(var j=0; j<temp.length;j++ ){//check duplicates
            if(temp[j]==this[i])//skip if already present 
               continue label;      
        }
        temp[temp.length] = this[i];
  }
  return temp;
 }

(But not a tested code!)

edited May 25 '13 at 11:29

answered May 25 '13 at 08:35

Grijesh Chauhan

57,103
20
141
208

1

This function doesn't just remove duplicates, it also sorts. Why encumber Array's prototype with such a specific function (which wouldn't work for any type of array as most can't be directly sorted) ? – Denys Séguret May 25 '13 at 08:37
Maybe better to test for inequality and dispose the `continue` – Matanya May 25 '13 at 08:41
@Matanya I am improving my answer – Grijesh Chauhan May 25 '13 at 08:42
http://stackoverflow.com/q/9229645/989121 – georg May 25 '13 at 09:37
@thg435 I had simple error of `{` in first code that's why it was not working, now corrected and tested. – Grijesh Chauhan May 25 '13 at 11:30
@GrijeshChauhan: sorry, it was meant as a comment to the question, not to your answer. – georg May 25 '13 at 11:32
how to use these functions, where to pass array which needs to be sorted – Phaneendra Charyulu Kanduri Mar 26 '16 at 15:43

Delete duplicate elements from an array

5 Answers5

Linked