Questions tagged [duplicates]

The "duplicates" tag concerns detecting and/or dealing with multiple instances of items in collections.

A duplicate is any re-occurrence of an item in a collection. This can be as simple as two identical strings in a list of strings, or multiple complex objects which are treated as the same object when compared to each other.

This tag may pertain to questions about preventing, detecting, removing, or otherwise dealing with unwanted duplicates, or adapting to safely allow duplicates.

15777 questions
2398
votes
39 answers

Finding duplicate values in a SQL table

It's easy to find duplicates with one field: SELECT email, COUNT(email) FROM users GROUP BY email HAVING COUNT(email) > 1 So if we have a table ID NAME EMAIL 1 John asd@asd.com 2 Sam asd@asd.com 3 Tom asd@asd.com 4 Bob …
Alex
  • 34,581
  • 26
  • 91
  • 135
2320
votes
54 answers

Remove duplicate values from JS array

I have a very simple JavaScript array that may or may not contain duplicates. var names = ["Mike","Matt","Nancy","Adam","Jenny","Nancy","Carl"]; I need to remove the duplicates and put the unique values in a new array. I could point to all the code…
kramden88
  • 23,477
  • 3
  • 19
  • 17
1452
votes
23 answers

LINQ's Distinct() on a particular property

I am playing with LINQ to learn about it, but I can't figure out how to use Distinct when I do not have a simple list (a simple list of integers is pretty easy to do, this is not the question). What I if want to use Distinct on a List on…
Patrick Desjardins
  • 136,852
  • 88
  • 292
  • 341
1435
votes
57 answers

Removing duplicates in lists

How can I check if a list has any duplicates and return a new list without duplicates?
Neemaximo
  • 20,031
  • 12
  • 32
  • 40
1371
votes
43 answers

How can I remove duplicate rows?

I need to remove duplicate rows from a fairly large SQL Server table (i.e. 300,000+ rows). The rows, of course, will not be perfect duplicates because of the existence of the RowID identity field. MyTable RowID int not null identity(1,1) primary…
Seibar
  • 68,705
  • 38
  • 88
  • 99
1015
votes
31 answers

How do I remove duplicates from a list, while preserving order?

How do I remove duplicates from a list, while preserving order? Using a set to remove duplicates destroys the original order. Is there a built-in or a Pythonic idiom?
Josh Glover
  • 25,142
  • 27
  • 92
  • 129
882
votes
77 answers

How to remove all duplicates from an array of objects?

I have an object that contains an array of objects. obj = {}; obj.arr = new Array(); obj.arr.push({place:"here",name:"stuff"}); obj.arr.push({place:"there",name:"morestuff"}); obj.arr.push({place:"there",name:"morestuff"}); I'm wondering what is…
Travis
  • 10,192
  • 4
  • 19
  • 19
769
votes
25 answers

Fastest way to duplicate an array in JavaScript - slice vs. 'for' loop

In order to duplicate an array in JavaScript: Which of the following is faster to use? Slice method var dup_array = original_array.slice(); For loop for(var i = 0, len = original_array.length; i < len; ++i) dup_array[i] = original_array[i]; I…
Marco Demaio
  • 33,578
  • 33
  • 128
  • 159
738
votes
28 answers

Find duplicate records in MySQL

I want to pull out duplicate records in a MySQL Database. This can be done with: SELECT address, count(id) as cnt FROM list GROUP BY address HAVING cnt > 1 Which results in: 100 MAIN ST 2 I would like to pull it so that it shows each row that…
Chris Bartow
  • 14,873
  • 11
  • 43
  • 46
690
votes
44 answers

How do I find the duplicates in a list and create another list with them?

How do I find the duplicates in a list of integers and create another list of the duplicates?
MFB
  • 19,017
  • 27
  • 72
  • 118
688
votes
17 answers

Removing duplicate rows in Notepad++

Is it possible to remove duplicated rows in Notepad++, leaving only a single occurrence of a line?
Przemysław Michalski
  • 9,627
  • 7
  • 31
  • 37
682
votes
7 answers

Find duplicate lines in a file and count how many time each line was duplicated?

Suppose I have a file similar to the following: 123 123 234 234 123 345 I would like to find how many times '123' was duplicated, how many times '234' was duplicated, etc. So ideally, the output would be like: 123 3 234 2 345 1
user839145
  • 6,883
  • 3
  • 16
  • 10
625
votes
32 answers

Remove duplicates from a List in C#

Anyone have a quick method for de-duplicating a generic List in C#?
JC Grubbs
  • 39,191
  • 28
  • 66
  • 75
573
votes
40 answers

How do I remove repeated elements from ArrayList?

I have an ArrayList, and I want to remove repeated strings from it. How can I do this?
user25778
  • 5,901
  • 3
  • 20
  • 12
568
votes
28 answers

How to delete duplicate rows in SQL Server?

How can I delete duplicate rows where no unique row id exists? My table is col1 col2 col3 col4 col5 col6 col7 john 1 1 1 1 1 1 john 1 1 1 1 1 1 sally 2 2 2 2 2 2 sally 2 2 2 2 2 2 I…
Fearghal
  • 10,569
  • 17
  • 55
  • 97
1
2 3
99 100