Remove duplicate inner lists from a list of list

Question

I apologize for the ambiguous title. I couldn't keep it clear and concise at the same time. So feel free to change it.

I have a big List which contains several other Lists. And these inner Lists contain Column objects.

List<List<Column>> listOfAllColumns;

Let's say my inner lists contain different Column objects like this:

list1 = {c1, c1, c2}
list2 = {c1, c2, c1}
list3 = {c2, c3}
list4 = {c1,c1, c2}

And the big list contains these lists: listOfAllColumns = {list1, list2, list3, list4}

Now I want a method that removes duplicate lists from the listOfAllColumns list. For example, it will look into the list above and remove list4.

list1: c1,c1,c2
list2: c1,c2,c1
list3: c2,c3
list4: c1,c1,c2 (it is equal to list1 so it is a duplicate)

Here is my code:

public class ColumnList
{
    public void RemoveDuplicateColumnTypes()
    {

        Column c1 = new Column() { SectionName = "C50", StirrupType = "Tie" };
        Column c2 = new Column() { SectionName = "C50", StirrupType = "Spiral" };
        Column c3 = new Column() { SectionName = "C40", StirrupType = "Tie" };


        List<Column> list1 = new List<Column>() { c1, c1, c2 };
        List<Column> list2 = new List<Column>() { c1, c2, c1 };
        List<Column> list3 = new List<Column>() { c2, c3 };
        List<Column> list4 = new List<Column>() { c1, c1, c2 };


        List<List<Column>> listOfAllColumns = new List<List<Column>>() { list1, list2, list3, list4 };

        var result = listOfAllColumns.Distinct();

    }
}

class Column
{
    public string SectionName;
    public string StirrupType;
    public int StirrupSize;
    public double StirrupSpacing;
}

By the way the order is important, so for example {c1, c2, c1} is different than {c2,c1,c1}.

Sounds like you need to look into the overload for Distinct, so you can pass in your comparator object. — gunr2171, Mar 05 '14 at 20:32
@RBarryYoung Now I want a method that removes duplicate lists from the listOfAllColumns list. — Vahid, Mar 05 '14 at 20:34
Hi, see http://stackoverflow.com/questions/3834461/removing-duplicates-from-a-list-c-sharp?rq=1 — Gus, Mar 05 '14 at 20:39

Servy · Answer 1 · 2014-11-17T21:49:15.803

5

What you need is an IEqualityComparer that is able to compare different sequences. This isn't terribly hard, given that it also has a way of comparing the items within it:

public class SequenceComparer<T> : IEqualityComparer<IEnumerable<T>>
{
    private IEqualityComparer<T> comparer;
    public SequenceComparer(IEqualityComparer<T> comparer = null)
    {
        this.comparer = comparer ?? EqualityComparer<T>.Default;
    }
    public bool Equals(IEnumerable<T> x, IEnumerable<T> y)
    {
        return x.SequenceEqual(y, comparer);
    }

    public int GetHashCode(IEnumerable<T> sequence)
    {
        unchecked
        {
            int hash = 19;
            foreach (var item in sequence)
                hash = hash * 79 + comparer.GetHashCode(item);
            return hash;
        }
    }
}

Now all you need to do is create an IEqualityComparer<Column> that is capable of comparing two column objects (through whatever meaningful way you want, presumably the default implementation is not sufficient).

Then you can just pass those to Distinct:

var query = listOfAllColumns.Distinct(
    new SequenceComparer<Column>(new ColumnComparer()));

edited Nov 17 '14 at 21:49

answered Mar 05 '14 at 20:36

Servy

202,030
26
332
449

Thanks. What does int hash = 19; do here? – Vahid Mar 05 '14 at 21:00
@Vahid Initializes the hash variable to a number, specifically, the number 19. While technically any number will be valid, having a prime number decreases the odds of collisions, which is nice, but certainly not a *requirement*. – Servy Mar 05 '14 at 21:01
Using Distinct() is the only path to take? Can it be done using Linq? – Vahid Mar 05 '14 at 21:04
@Vahid `Disctinct` *is* a LINQ method, so this *is* using LINQ. And no, `Distinct` is not the only method for solving this problem, there are always other ways, but `Distinct` likely the easiest option. – Servy Mar 05 '14 at 21:06
ColumnComparer() is not recognized. I couldn't understand this part. – Vahid Mar 05 '14 at 21:25
1

@Vahid You need to define a comparer for your `Column` class based on how you want to define equality between two columns. It is where you specify when two different columns are considered equal, assuming the `Equals` and `GetHashCode` methods of the class itself do not already define the appropriate behavior. If they already do, you can omit that call. – Servy Mar 05 '14 at 21:28
I guess I need to study and increase my knowledge of the IEqualityComparer. Thank you so much. – Vahid Mar 05 '14 at 21:32
1

@Vahid: if you don't know what to do, have a look at my (working) implementation below. Instead of overriding Equals+GetHashCode you could use it in an `IEqualityComparer` as Servy has suggested. – Tim Schmelter Mar 05 '14 at 21:33
@TimSchmelter Thanks Tim. Actually I just tested that but I couldn't get it to work. – Vahid Mar 05 '14 at 21:37
1

@Vahid: i have tested it within your `RemoveDuplicateColumnTypes` and it returned the first three lists and removed the fourth. Maybe you have not executed thbe query. Note that `Distinct` is implemented using deferred execution. You could execute it for example by adding `ToList()`. So to remove them from your list you need to reassign it: `listOfAllColumns = listOfAllColumns.Distinct(new ColumnListComparer()).ToList();` – Tim Schmelter Mar 05 '14 at 21:40
@TimSchmelter Thanks. It worked, now I really need to get to study these. I feel like a cargo cult programmer now! :p – Vahid Mar 05 '14 at 21:49

Tim Schmelter · Accepted Answer · 2014-03-05T22:09:11.130

I would implement a custom IEqualityComparer<IEnumerable<Column>> which you can use for Distinct:

public class ColumnListComparer : IEqualityComparer<IEnumerable<Column>>
{
    public bool Equals(IEnumerable<Column> x, IEnumerable<Column> y)
    {
        if (x == null || y == null) return false;
        if (object.ReferenceEquals(x, y)) return true;
        return x.SequenceEqual(y);
    }

    public int GetHashCode(IEnumerable<Column> obj)
    {
        unchecked
        {
            int hash = 17;
            foreach(Column col in obj)
            {
                hash = hash * 23 + (col == null ? 0 : col.GetHashCode());
            }
            return hash;
        }
    }
}

Now this works:

var result = listOfAllColumns.Distinct(new ColumnListComparer());

You also need to override Equals + GetHashCode in your class Column:

public class Column
{
    public string SectionName;
    public string StirrupType;
    public int StirrupSize;
    public double StirrupSpacing;

    public override bool Equals(object obj)
    {
        Column col2 = obj as Column;
        if(col2 == null) return false;
        return SectionName    == col2.SectionName
            && StirrupType    == col2.StirrupType 
            && StirrupSize    == col2.StirrupSize
            && StirrupSpacing == col2.StirrupSpacing;
    }

    public override int GetHashCode()
    {
        unchecked
        {
            int hash = 17;
            hash = hash * 23 + (SectionName ?? "").GetHashCode();
            hash = hash * 23 + (StirrupType ?? "").GetHashCode();
            hash = hash * 23 + StirrupSize.GetHashCode();
            hash = hash * 23 + StirrupSpacing.GetHashCode();
            return hash;
        }
    }
}

Remove duplicate inner lists from a list of list

2 Answers2

Linked

Related