Get a list of duplicates from list based on properties comparison

Question

I have a List<Demo>

public class Demo
{
    public Demo()
    {

    }

    public int Id { get; set; }
    public string Name { get; set; }
    public string Title { get; set; }
}

The Id property is unique for every record in the list.

How can I get a List<Demo> from the original List<Demo> with all the duplicates where the Name and Title are same.

What I did so far but what I get is one single record:

List<Demo> demo = new List<Demo>();

demo.Add(new Demo()
{
    Id = 1,
    Name = "Demo",
    Title = "A"
});

demo.Add(new Demo()
{
    Id = 2,
    Name = "Demo",
    Title = "A"
});

demo.Add(new Demo()
{
    Id = 3,
    Name = "Demo",
    Title = "A"
});

demo.Add(new Demo()
{
    Id = 4,
    Name = "Demo1",
    Title = "A"
});

demo.Add(new Demo()
{
    Id = 5,
    Name = "Demo2",
    Title = "A"
});

and then i am doing:

var duplicates = demo.GroupBy(t => new { t.Name, t.Title })
                     .Where(t => t.Count() > 1)
                     .Select(g => g.Key).ToList();

From the example above I should get a List<Demo> with the first 3 items where the ids are:1,2,3 because Name and Title are same.

[Using LINQ to find duplicates across multiple properties](https://stackoverflow.com/questions/5596604/using-linq-to-find-duplicates-across-multiple-properties) — Lei Yang, Jul 06 '17 at 06:38
Yes, you'll only get one result - but that's a group with three elements. Bear in mind that you could have multiple groups (e.g. multiple "Demo2/A" records). It's not clear what you expect in that situation. — Jon Skeet, Jul 06 '17 at 06:41
@user2818430 what do you mean? if you tried using the link's method then paste your latest code. — Lei Yang, Jul 06 '17 at 06:41
Possible duplicate of [Using LINQ to find duplicates across multiple properties](https://stackoverflow.com/questions/5596604/using-linq-to-find-duplicates-across-multiple-properties) — Adwaenyth, Jul 06 '17 at 06:41
@JonSkeet: The list should include all duplicates. If there are multiple Demo2/A those should be included in the list also — user2818430, Jul 06 '17 at 06:42
So you'd want a *single* list that included all Demo/A elements and all Demo2/A elements? — Jon Skeet, Jul 06 '17 at 06:43

score 14 · Accepted Answer · answered Jul 06 '17 at 06:44

14

It sounds like all you're missing is a SelectMany call. Currently you're creating all the appropriate groups and filtering down to groups with more than one entry - but if you want a single flat list, you need to flatten those groups back to their elements:

var duplicates = demo
    .GroupBy(t => new { t.Name, t.Title })
    .Where(t => t.Count() > 1)
    .SelectMany(x => x) // Flatten groups to a single sequence
    .ToList();

Note that this doesn't mean every entry in the resulting list will have the same name and title. It does mean that every entry will have a name/title combination in common with at least one other entry.

answered Jul 06 '17 at 06:44

Jon Skeet

1,421,763
867
9,128
9,194

can you still advise how to ignore empty strings? if for example I have records where Name and Title are empty strings – user2818430 Jul 06 '17 at 07:12
1

@user2818430: You should ask that as a separate question, after trying it for yourself. (Hint: filter them right at the start with a `Where` clause.) – Jon Skeet Jul 06 '17 at 08:36

Harald Coppoolse · Answer 2 · 2017-07-06T12:11:59.177

You were almost there.

Groupby divides a sequence into groups. Each group has something in common: the Key of the group. Each group is a sequence containing all elements in the original sequence that match the key.

Your key is new { t.Name, t.Title }. Your original sequence will be divided into groups of object that had the same name / title:

Group Demo / A contains three elements: Id = 1, Id = 2, Id = 3
Group Demo1 / A contains only element Id = 4
Group Demo2 / A contains only element Id = 5

The Where after you group returns a sequence of IGrouping. This sequence contains only one element: the only group that has more than one element. This is the group with key Demo/A.

Your specification was not a sequence of IGrouping (where each group is a sequence of Demo), but one list of Demo that contains all elements with duplicate Name/Title.

This means you have to take the elements from all groups after your Where (in your example this sequence contains only one group) and concatenate all these groups into one sequence. This is done by Enumerable.SelectMany

IEnumerable<Demo> duplicates = demo
    .GroupBy(demoElement => new {demoElement.Name, demoElement.Title})
    .Where(group => group.Skip(1).Any())
    .SelectMany(group => group);

The SelectMany takes the sequence of each group and concatenates all sequences into one sequence.

By the way, did you notice that I didn't use Count() to detect if there are duplicates, but Skip(1).Any(). If one of your groups would have hundreds of elements, it would be enough to stop counting after the first element. It is a waste of computing power to count all elements to detect if there is more than one.

One final hint: don't use ToList() if you are not certain you need it. If the user of your piece of code only wants the first element, or the first few, it is a waste if you calculate all elements into a list. Keep it IEnumerable<Demo> as long as possible.

Nikhil Agrawal · Answer 3 · 2017-07-06T06:53:43.970

Hers's what you want if you have more than 1 title and names are same. Say the last object is Demo1 and A.

var duplicates = demo.GroupBy(t => new { t.Name, t.Title }).Where(t => t.Count() > 1)
                     .Select(x => new { Count = x.Count(), Values = x.Select(y => y)})
                     .ToList();

What this does it, gives you count and all the grouped values (based on Name and Title) and their Count.

score 0 · Answer 4 · answered Jul 06 '17 at 09:25

0

Below code can help you ,

List duplicates = demo.GroupBy(grp => new { grp.Name, grp.Title }).SelectMany(selm => selm.Skip(1)).Distinct().ToList();

It is working fine as per your expected answer.

answered Jul 06 '17 at 09:25

Pramod Sutar

70
1
5

Get a list of duplicates from list based on properties comparison

4 Answers4