Test if all values in a list are unique

Question

I have a small list of bytes and I want to test that they're all different values. For instance, I have this:

List<byte> theList = new List<byte> { 1,4,3,6,1 };

What's the best way to check if all values are distinct or not?

Answer 1

bool isUnique = theList.Distinct().Count() == theList.Count();

Answer 2

Here's another approach which is more efficient than Enumerable.Distinct + Enumerable.Count (all the more if the sequence is not a collection type). It uses a HashSet<T> which eliminates duplicates, is very efficient in lookups and has a count-property:

var distinctBytes = new HashSet<byte>(theList);
bool allDifferent = distinctBytes.Count == theList.Count;

or another - more subtle and efficient - approach:

var diffChecker = new HashSet<byte>();
bool allDifferent = theList.All(diffChecker.Add);

HashSet<T>.Add returns false if the element could not be added since it was already in the HashSet. Enumerable.All stops on the first "false".

Answer 3

Okay, here is the most efficient method I can think of using standard .Net

using System;
using System.Collections.Generic;

public static class Extension
{
    public static bool HasDuplicate<T>(
        this IEnumerable<T> source,
        IEqualityComparer<T> comparer,
        out T firstDuplicate)
    {
        ArgumentNullException.ThrowIfNull(source);
        ArgumentNullException.ThrowIfNull(comparer);
        
        (bool result, firstDuplicate) = HasDuplicateImplementation(source, comparer);
        
        return result;
    }
    
    public static bool HasDuplicate<T>(
        this IEnumerable<T> source,
        out T firstDuplicate)
    {
        ArgumentNullException.ThrowIfNull(source);
        var comparer = EqualityComparer<T>.Default;
        
        (bool result, firstDuplicate) = HasDuplicateImplementation(source, comparer);
        
        return result;
    }
    
    private static (bool, T) HasDuplicateImplementation<T>(
        IEnumerable<T> source,
        IEqualityComparer<T> comparer)
    {
        var checkBuffer = new HashSet<T>(comparer);
        foreach (var t in source)
        {
            if (!checkBuffer.Add(t))
            {
                return (true, t);
            }
        }
        
        return (false, default);
    }
}

essentially, what is the point of enumerating the whole sequence twice if all you want to do is find the first duplicate.

Answer 4

The similar logic to Distinct using GroupBy:

var isUnique = theList.GroupBy(i => i).Count() == theList.Count;

Answer 5

I check if an IEnumerable (aray, list, etc ) is unique like this :

var isUnique = someObjectsEnum.GroupBy(o => o.SomeProperty).Max(g => g.Count()) == 1;

Answer 6

One can also do: Use Hashset

var uniqueIds = new HashSet<long>(originalList.Select(item => item.Id));

            if (uniqueIds.Count != originalList.Count)
            {
            }

Answer 7

There are many solutions.

And no doubt more beautiful ones with the usage of LINQ as "juergen d" and "Tim Schmelter" mentioned.

But, if you bare "Complexity" and speed, the best solution will be to implement it by yourself. One of the solution will be, to create an array of N size (for byte it's 256). And loop the array, and on every iteration will test the matching number index if the value is 1 if it does, that means i already increment the array index and therefore the array isn't distinct otherwise i will increment the array cell and continue checking.

Answer 8

And another solution, if you want to find duplicated values.

var values = new [] { 9, 7, 2, 6, 7, 3, 8, 2 };

var sorted = values.ToList();
sorted.Sort();
for (var index = 1; index < sorted.Count; index++)
{
    var previous = sorted[index - 1];
    var current = sorted[index];
    if (current == previous)
        Console.WriteLine(string.Format("duplicated value: {0}", current));
}

Output:

duplicated value: 2
duplicated value: 7

http://rextester.com/SIDG48202