Remove duplicates while merging lists using Union in LINQ

Question

I am trying to merge two lists using list.Union in LinqPad but I can't get it to work and wanted to check my understanding is correct.

Given this simple class:

public class Test 
{
   public int Id { get; set;}
   public int field1 { get; set; }

   public bool Equals(Test other)
   {        
      return this.Id.Equals(other.Id);
   }
}

And two lists populated like this:

List<Test> list = new List<Test>();
list.Add( new Test { Id = 1, field1 = 1});
list.Add( new Test { Id = 1, field1 = 2});
list.Add( new Test { Id = 2, field1 = 3});
list.Add( new Test { Id = 2, field1 = 4});

List<Test> list2 = new List<Test>();
list2.Add( new Test { Id = 1, field1 = 1});
list2.Add( new Test { Id = 1, field1 = 2});
list2.Add( new Test { Id = 2, field1 = 3});
list2.Add( new Test { Id = 2, field1 = 4});

I then try: var mergedList = list.Union(list2).ToList(); and output the data using a simple foreach loop and get this output:

ID: 1 -------- 1
ID: 1 -------- 2
ID: 2 -------- 3
ID: 2 -------- 4
ID: 1 -------- 1
ID: 1 -------- 2
ID: 2 -------- 3
ID: 2 -------- 4

I was under the impression that Union should remove the duplicates to return:

ID: 1 -------- 1
ID: 1 -------- 2
ID: 2 -------- 3
ID: 2 -------- 4

Am I doing something wrong or have I misunderstood?

Also, should it work without explicitly overriding the Equals method in the Test class?

Thanks

You should read [this documentation page](http://msdn.microsoft.com/en-us/library/system.object.gethashcode.aspx). — Jon, Jun 07 '13 at 09:20

score 20 · Accepted Answer · edited May 23 '17 at 10:31

20

In your case you simply define some method, that LINQ knows nothing about. It's like creating method bool HeyEquateMeWith(Test other) and expect, that LINQ will call it when doing set operations.

You need to define your class as following (override Object's Equals and GetHashCode methods):

public class Test 
{
   public int Id { get; set;}
   public int field1 { get; set; }  

   public override bool Equals(object other) //note parameter is of type object
   {        
        Test t = other as Test;
        return (t != null) ? Id.Equals(t.Id) : false;
   }

   public override int GetHashCode()
   {
        return Id.GetHashCode();
   }
}

Now Union will call your overridden Equals and GetHashCode methods. Also you should ALWAYS override GetHashCode when you override Equals method.

edited May 23 '17 at 10:31

Community

1
1

answered Jun 07 '13 at 09:24

Ilya Ivanov

23,148
4
64
90

Thanks. Yeah, I thought I could leave the GetHashCOde for an example but it is required. How does the default comparer work for Union or do I always need to override Equals and GethashCode? – davy Jun 07 '13 at 09:44
2

@davy default comparer will look at object identity (same functionality as `==` operator). If you change your `Test` class to `struct`, the default equality will have value semantics (compare contents of two structs including `Id` and `field1`) – Ilya Ivanov Jun 07 '13 at 09:47

score 2 · Answer 2 · answered Dec 02 '15 at 18:05

You can try something like that if not happy with default comparer (which is, in turn, utilizes GetHashCode method as @IlyaIvanov had mentioned):

// get all items that "other than in first list", so Where() and Any() are our filtering expressions
var delta = list2.Where(x2 => !list.Any(x1 => (x1.Id == x2.Id) && (x1.field1 == x2.field1)));

// now let merge two enumerables that have nothing "equal" between them
var merged = list.Union(delta).ToList();

score 1 · Answer 3 · answered Jun 07 '13 at 09:40

1

You can create a class implementing

IEqualityComparer<Test>

Is this class define Equals and GetHashCode After it you can pass this comparer to you Union method Just like that:

public class MyComparer:IEqualityComparer<Test>{
//Equals and GetHashCode
}

var mergedList = list.Union(list2, new MyComparer()).ToList();

answered Jun 07 '13 at 09:40

N F

31
2

This will work for LINQ to Objects but is not supported for LINQ to Entities. I found this out the hard way. – Suncat2000 Jun 24 '20 at 14:51

score 1 · Answer 4 · answered May 13 '16 at 18:16

Just want to leave this here for anyone that still couldn't get it. I found this article super helpful about the compare class that inherits from IEqualityComparer http://alicebobandmallory.com/articles/2012/10/18/merge-collections-without-duplicates-in-c

Remove duplicates while merging lists using Union in LINQ

4 Answers4

Linked