Deserialization without duplicates

Question

I'm deserializing JSON into class objects and I would like to avoid duplicate ones. I've tried to implement IEqualityComparer, but it does not seem to work, because Equals() or GetHashCode() methods are never fired and resulting HashSet contains duplicates too. What's I'm doing wrong?

[DataContract]
        public class Release : IEqualityComparer<Media>
        {
            public bool Equals(Media m1, Media m2)
            {
                if (m1.mediaFormat == m2.mediaFormat && m1.getTrackPosition() == m2.getTrackPosition()) { Console.WriteLine("Duplicate!"); return true; }
                else return false;
            }

            public int GetHashCode(Media m) { Console.WriteLine("DDD!");  return m.ToString().GetHashCode(); }

            [DataMember(Name = "title")]
            public string releaseTitle { get; set; }
            [DataMember(Name = "date")]
            public string releaseDate { get; set; }
            [DataMember(Name = "id")]
            public string releaseMBID; //required for fetching artwork
            [DataMember(Name = "media")]
            public HashSet<Media> releaseMedia { get; set; }
            public List<Media> getReleaseMedia() { return releaseMedia.ToList<Media>(); }
        }

[DataContract]
        public class Media
        {

            [DataMember(Name = "format")]
            public string mediaFormat { get; set; }
            [DataMember(Name = "track-offset")]
            public string mediaTrackOffset { get; set; }
            [DataMember(Name = "track-count")]
            public string mediaTrackCount { get; set; }
            public string getTrackPosition() { return (Convert.ToInt32(mediaTrackOffset) + 1).ToString() + "/" + mediaTrackCount; }
        }

Could we have context? You are getting a Json string, you are deserilizing it.Deserializing to `List` or `Hashset<>`? And there is duplicate in this collection. But did you apply `Distinct` on the list to clear the duplicate? Or do you want to deserilize the object one by one and check for duplicate befor adding them to the collection? — Drag and Drop, Feb 16 '21 at 10:32
I get JSON from remote server and directly deserialize it into the class. That JSON contains some lists, which might have duplicates, so I'm trying to use HashSet for them — Maxim Voloshin, Feb 16 '21 at 10:36
`Release` implements `IEqualityComparer` which means you could give it to something that wants an `IEqualityComparer` (like a `HashSet`) and have it use this as their equality comparer. However you don't seem to pass it to anything, resulting in it not being used. Are you sure you don't instead want to have `Media` implement `IEquatable`? — , Feb 16 '21 at 10:39
Normally there are 2 routes you can take. First you can define `Media` as `Media: IEquatable` letting you generally define equality for all `Media` objects. Second route is to make a custom `IEqualityComparer` (I would strongly advise you to do this in a seperate class instead of in `Release`), you can then pass that to whatever you want to use this specific `IEqualityComparer` instead of the standard equality (by default for classes uses to the `object` equality so it's equality by reference). If you take option 2 then yes you need to manually pass it to the `HashSet` — , Feb 16 '21 at 10:51
https://stackoverflow.com/questions/8952003/how-does-hashset-compare-elements-for-equality. And exactly what knoop said. I always get tripped by IEqualityComparer vs IEquatable. Sorry for the missleading comment. — Drag and Drop, Feb 16 '21 at 10:57
I've implemented iequatable for media, Equals still never fired — Maxim Voloshin, Feb 16 '21 at 11:18
Then you've probably implemented it incorrectly. If no `IEqualityComparer` is passed to the `HashSet` it should automatically pick up on the `IEquatable` implementation. A common mistake however is to have an incorrect `GetHashCode`. A `HashSet` first checks if it has any entries with the same `GetHashCode` value, and only if it finds a match will it check equality for those! Small fiddle to show how it works: https://dotnetfiddle.net/WKscac — , Feb 16 '21 at 13:55

Self · Answer 1 · 2021-02-16T12:23:59.307

Yes you will need to pass the IEqualityComparer to the constructor like :

public class HFoo{  
    public string Name {get;set;}
    public HashSet<Pet> Pets {get;set;} = new HashSet<Pet>(new CustomComparer());
}

public class Pet{
}

public class CustomComparer : IEqualityComparer<Pet>{
    public int GetHashCode(Pet obj) {
        return 1;
    }       
    public bool Equals(Pet obj1, Pet obj2) {        
        return true;
    }
}

Or the default object conparer will be used.

As you can see in the following demo: https://dotnetfiddle.net/Y9Afem.
The IEquatable will not be call.

Thank you! I'll try that solution – Maxim Voloshin Feb 16 '21 at 12:37 — Maxim Voloshin, Feb 16 '21 at 12:37

Deserialization without duplicates

1 Answers1