Duplicate by using LINQ with mulitiple OR creteria

Question

I can retrieve duplicate by using this query:

var duplicates = grpDupes
    .GroupBy(i => new { i.Email })
    .Where(g => g.Count() > 1)
    .SelectMany(g => g);

But i am interested to find duplicates by using either Email or Address or xyz. If i modify above query

GroupBy(i => new { i.Email, i.Address })

then it becomes AND condition, any help?

AcidJunkie · Answer 1 · 2014-05-14T11:35:28.750

You have to use the overloaded method which accepts an EqualityComparer.

    /// <summary>
/// Factory class which creates an EqualityComparer based on lambda expressions.
/// </summary>
/// <typeparam name="T">The type of which a new equality comparer is to be created.</typeparam>
public static class EqualityComparerFactory<T>
{
    private class MyComparer : IEqualityComparer<T>
    {
        private readonly Func<T, int> _getHashCodeFunc;
        private readonly Func<T, T, bool> _equalsFunc;

        public MyComparer(Func<T, T, bool> equalsFunc, Func<T, int> getHashCodeFunc = null)
        {
            _getHashCodeFunc = getHashCodeFunc ?? (a=>0);
            _equalsFunc = equalsFunc;
        }

        public bool Equals(T x, T y)
        {
            return _equalsFunc(x, y);
        }

        public int GetHashCode(T obj)
        {
            return _getHashCodeFunc(obj);
        }
    }

    /// <summary>
    /// Creates an <see cref="IEqualityComparer{T}" /> based on an equality function and optionally on a hash function.
    /// </summary>
    /// <param name="equalsFunc">The equality function.</param>
    /// <param name="getHashCodeFunc">The hash function.</param>
    /// <returns>
    /// A typed Equality Comparer.
    /// </returns>
    public static IEqualityComparer<T> CreateComparer(Func<T, T, bool> equalsFunc, Func<T, int> getHashCodeFunc = null)
    {
        ArgumentValidator.NotNull(() => equalsFunc);

        return new MyComparer(equalsFunc, getHashCodeFunc);
    }
}

Sample Usage:

        var comparer = EqualityComparerFactory<YourClassHere>.CreateComparer((a, b) => a.Address == b.Address || a.Email == b.Email);

        data.GroupBy(a => a, comparer);

It will throw `ArgumentNullException` without hashcode method isn't it? — Sriram Sakthivel, May 14 '14 at 11:09
Thanks there are two issues , ArgumentValidator does not exist in current context does not take 2 paramters var comparer = EqualityComparerFactory.CreateComparer(null, (a, b) => a.Address == b.Address || a.Email == b.Email ); — user3636316, May 14 '14 at 11:33
Sorry for that one. this one originates from a framework i am using.
To the second issue: Just leave out the null argument. — AcidJunkie, May 14 '14 at 11:34

Tim Schmelter · Accepted Answer · 2014-05-14T12:34:49.253

2

You could use EXISTS in SQL which is Any in LINQ:

var duplicates = grpDupes
    .Where(i => (i.Email.Trim() != "" || i.Address.Trim() != "")  && grpDupes
        .Any(i2 => i.ID != i2.ID && 
            ((i.Email.Trim()   != "" && i.Email   == i2.Email) || 
             (i.Address.Trim() != "" && i.Address == i2.Address))));

Note that i've used ID as the primary key column. If you don't have one you need to use the column(s) that you want to use as identifier.

If you use as database driven LINQ provider like LINQ-To-SQL or LINQ-To-Entities this is efficient.

edited May 14 '14 at 12:34

answered May 14 '14 at 11:09

Tim Schmelter

450,073
74
686
939

This seems working , anyone tell me why should i not use this? – user3636316 May 14 '14 at 11:38
can we improve this by ignoring blank emails or address in i.Email == i2.Email || i.Address == i2.Address). If emails in both records r empty then it also consider it as duplicate – user3636316 May 14 '14 at 12:22
@user3636316: i have edited my answer to provide a way that should work with all LINQ providers (as opposed to `String.IsNullOrWhiteSpace` for example). – Tim Schmelter May 14 '14 at 12:35

score 0 · Answer 3 · answered May 14 '14 at 13:01

I'd keep this very simple by using .ToLookup().

How about this?

var emailLookup = grpDupes.ToLookup(x => x.Email);
var addressLookup = grpDupes.ToLookup(x => x.Address);

var duplicates = grpDupes
    .Where(x =>
        emailLookup[x.Email].Count() > 1 || addressLookup[x.Address].Count() > 1);

Duplicate by using LINQ with mulitiple OR creteria

3 Answers3