Javascript: searching within an array for dupes

Question

This is what I am trying to do: I present the user with a textarea and he must enter some domains, if he enters the same domain twice (a duplicate) I want to delete the dupes.

So far I have come till the part where I can find the dupes, this is the code I am using:

function check_if_already_in_the_list___manual_textbox()
{

var therows=0;
var thetext = document.forms[0].text.value;
var newtext = thetext.split("\n");
therows+=newtext.length;
var i;
var match_counter=0;

    for(i=0;i<newtext.length;i++) // first iterate over the number of items
    {
        for(j=0;j<newtext.length;j++) // second, start a second loop to compare each other
        {

            if(newtext[j].toLowerCase()==newtext[i].toLowerCase())
            {           
            match_counter++;
            }

        if(match_counter >=2) // Found dupe!
        {alert("Matched:"+newtext[j]+" "+newtext[i]+" Counter"+match_counter);
match_counter=0;}


        }
        alert("Match counter:"+match_counter+ " D:"+newtext[i]);'
match_counter=0;
    }
//alert(""+match_counter);
return match_counter;
}

Any suggestions to do this better would be most appreciated, also I have no idea how to take out the dupes :(
Googling I see that I probably have to use "splice" but not really sure.

Thanks in advance!
R

(P.S Sorry the format looks weird, but that happened when I pasted in the code)

Joseph Marikle · Accepted Answer · 2012-01-06T13:08:21.017

2

Here's my feeble attempt at it. It's similar to

var arr = ["1","2","3","4","5","3","2","3","4"];
var arrCopy = [];
var list = {};

for (var i = 0, len = arr.length; i < len; i++) {
    if(!list[arr[i]])
        arrCopy.push(arr[i]);
    list[arr[i]] = ++list[arr[i]] | 0;
}

The object will also contain how many duplicates there were for each one. arrCopy has the unique values.

EDIT: see RobG's comment concerning hasOwnProperty. In this case it should be

...
    if(!list.hasOwnProperty(arr[i]))
...

edited Jan 06 '12 at 13:08

answered Jan 06 '12 at 00:59

Joseph Marikle

76,418
17
112
129

3

Good, but you should use `list.hasOwnProperty(arr[i])` since the strings are arbitrary and there is a chance (though possibly not very high with domain names) that one may match a built–in Object method, e.g. `var arr = ['toString','valueOf'];` -> `arrCopy.length = 0`. – RobG Jan 06 '12 at 03:02

score 2 · Answer 2 · answered Jan 06 '12 at 02:24

2

Lots of answers. Here's one that uses a generic funciton to create an array of unique members. Note that the result will be sorted, using an object and getting back properties using for..in isn't guaranteed to maintain any particular order.

var el = document.forms[0].text;
el.value = unique(el.value.toLowerCase().split(/[\n\r]/)).join('\n');

function unique(arr) {
  arr.sort();
  var i = arr.length;

  while (i--) {
    if (arr[i] == arr[i - 1]) {
       arr.splice(i, 1);
    }
  }
  return arr;
}

answered Jan 06 '12 at 02:24

RobG

142,382
31
172
209

Joseph Marikle's answer uses an object to check for duplicates but returns a new array that keeps exactly the order of the original array (acting on the assumption that where there are duplicates the _first_ one is the one that is retained in the new array). – nnnnnn Jan 06 '12 at 02:45
1

Yes, but it doesn't use for..in, though it really should use a *hasOwnProperty* filter since the strings to be tested are arbitrary. ;-) – RobG Jan 06 '12 at 02:55
Upvoted your answer, would pick it but I already used the other answer as my code, will keep yours as a back up (upvoted your comments as well) Thanks! – Ryan Jan 06 '12 at 12:17
@Ryan You can still change the accepted answer to RobG's. I won't mind. :) Plus Rob's answer looks a whole lot more solid and less hackish than mine. :P I think it's important to have the actual "best answer" marked as the solution so future developers who come across the question have the best answer most prominently available. :) – Joseph Marikle Jan 06 '12 at 13:13
Joseph—your answer is fine as the accepted answer, I was just offerring an alternative using a generic unique function, otherwise I'd have suggested what you had with a *hasOwnProperty* test. – RobG Jan 09 '12 at 00:08

Krishna Bharadwaj · Answer 3 · 2012-01-06T01:00:55.280

1

You can make use of something called as associative arrays to solve this problem. See if it works for you.

var initial_array = ['www.yahoo.com', 'www.google.com', 'www.facebook.com', 'www.google.com'];
var set = {};
for (var domain in initial_array){
    set[initial_array[domain].toLowerCase()] = true;
}
alert(set);

edited Jan 06 '12 at 01:00

answered Jan 06 '12 at 00:51

Krishna Bharadwaj

565
4
13

OP wants the count for how many dupes there. You method can remove the dupes but cannot count them. – Eric Yin Jan 06 '12 at 00:53
I don't mind removing the dupes without counting them, as long as there is one original left (it should NOT delete all instances!) – Ryan Jan 06 '12 at 00:57
@EricYin, I did not see count anywhere in the question(other than the code). Ryan is using count for detecting the duplicate and he is not interested in the number if I am not wrong :-) – Krishna Bharadwaj Jan 06 '12 at 00:59
Note that you shouldn't use a for..in loop on a JS array. – nnnnnn Jan 06 '12 at 02:16
@nnnnnn, I wasn't aware of that, can you provide some more info related to that? May be some articles on the internet? – Krishna Bharadwaj Jan 06 '12 at 02:55
1

Here's a SO question about it, with quite a lot of information in the answers: http://stackoverflow.com/q/500504/615754 (Sorry, I don't have time to look for external articles, but I'm sure you can find some with Google.) – nnnnnn Jan 06 '12 at 03:23
1

@KrishnaBharadwaj— for..in iteration on Arrays should not be used because 1) properties may not be returned in any particular order 2) it is reasonably common to add methods to Array.prototype that will be returned by the loop 3) if the special properteis of an Array aren't being used, then a plain Object should be used. However, there are still cases where for..in may be suitable, such as sparse arrays, provided action is taken to deal with the above issues. – RobG Jan 06 '12 at 03:26

score 0 · Answer 4 · answered Jan 06 '12 at 00:49

0

1st Copy array, and lowercase it first

2nd Array has a "sort" function in javascript, I suggest you sort it before compare

3rd I saw you are using bubble sorting, it's good. At for(j=0;j<newtext.length;j++), you do not need to start from 0, you can start from i

last, use something already there, http://api.jquery.com/jQuery.unique/

answered Jan 06 '12 at 00:49

Eric Yin

8,737
19
77
118

I cannot use jQuery, this is going into a firefox addon. Can you copy and make changes to my code so I can understand better please, I am not good with JS – Ryan Jan 06 '12 at 00:58

score 0 · Answer 5 · answered Jan 06 '12 at 01:04

I think there is a mistake here..

for(i=0;i<newtext.length;i++) // first iterate over the number of items
{
    for(j=0;j<newtext.length;j++) // second, start a second loop to compare each other

This code should not be??

for(i=0;i<newtext.length -1;i++) // first iterate over the number of items
{
    for(j=i+1;j<newtext.length;j++) // second, start a second loop to compare each other

note j=i+1 and newtext.length -1 (this last one is optional)

Then:

  if(newtext[j].toLowerCase()==newtext[i].toLowerCase())
      return 'dupe';

Javascript: searching within an array for dupes

5 Answers5