How to check duplicate string array in list?

Question

How to check duplicate string array in list? I declare string array list like this:

List<string[]> list = new List<string[]>();

and I add a few items in the list.

list.Add(new string[3] {"1","2","3"});
list.Add(new string[3] {"2","3","4"});
list.Add(new string[1] {"3"});
list.Add(new string[1] {"3"});
list.Add(new string[3] {"1","2","3"});

now I want to get to know which items are duplicated. I tried like below to add the duplicated items to new list:

for (int j = 0; j < list.Count - 1; j++)
{
    for (int k = list.Count - 1; k > j; k--)
    {
        if (j != k)
        {
            if (Enumerable.SequenceEqual(list[j], list[k]))
            {
                savedDistinctList.Add(list[j]);
            }
        }
    }
}

and finally I want to remove the duplicated item in the first list. so I want to see 3 items in the list.([1,2,3],[2,3,4],[3])

Perhaps any idea using LINQ or something else?

If you're saving the duplicates can't you just use list.Remove()? — Cyclone6664, Dec 20 '22 at 01:31
Does this answer your question? [C# LINQ find duplicates in List](https://stackoverflow.com/questions/18547354/c-sharp-linq-find-duplicates-in-list) — Skip, Dec 20 '22 at 01:39
See [IEqualityComparer for SequenceEqual](https://stackoverflow.com/questions/14675720/iequalitycomparer-for-sequenceequal) for an example of a `IEqualityComparer>` to use with [`Distinct`](https://learn.microsoft.com/en-us/dotnet/api/system.linq.enumerable.distinct?view=net-7.0#system-linq-enumerable-distinct-1(system-collections-generic-ienumerable((-0))-system-collections-generic-iequalitycomparer((-0)))). e.g. `var noduplicates = list.Distinct(new ArrayComparer());` — Jonathan Dodds, Dec 20 '22 at 01:47

score 0 · Answer 1 · answered Dec 20 '22 at 01:38

This has already been replied here: C# LINQ find duplicates in List by @Save

The easiest way to solve the problem is to group the elements based on their value, and then pick a representative of the group if there are more than one element in the group. In LINQ, this translates to:

var query = lst.GroupBy(x => x)
              .Where(g => g.Count() > 1)
              .Select(y => y.Key)
              .ToList();

If you want to know how many times the elements are repeated, you can use:

var query = lst.GroupBy(x => x)
              .Where(g => g.Count() > 1)
              .Select(y => new { Element = y.Key, Counter = y.Count() })
              .ToList();

This will return a List of an anonymous type, and each element will have the properties Element and Counter, to retrieve the information you need.

And lastly, if it's a dictionary you are looking for, you can use

var query = lst.GroupBy(x => x)
              .Where(g => g.Count() > 1)
              .ToDictionary(x => x.Key, y => y.Count());

This will return a dictionary, with your element as key, and the number of times it's repeated as value.

Apply with a foreach on your list.

var query = list.GroupBy(x => x).Where(g => g.Count() > 1).Select(y => y.Key).ToList(); I tried but the query count is 0. — Jacob, Dec 20 '22 at 02:01

Dmitry Bychenko · Accepted Answer · 2022-12-20T01:53:03.760

First we have to teach .Net how to compare arrays:

private sealed class ArrayEqualityComparer<T> : IEqualityComparer<T[]> {
  public bool Equals(T[] left, T[] right) {
    if (ReferenceEquals(left, right))
      return true;
    if (left is null || right is null)
      return false;

    return left.SequenceEqual(right);
  }

  public int GetHashCode(T[] array) => array is null
    ? -1
    : array.Length;
}

Then you can use Linq Distinct with this class implemented:

using System.Linq;

...

savedDistinctList = list
  .Distinct(new ArrayEqualityComparer<string>())
  .ToList();

If you want to modify the existing list, you can use HashSet<T>:

var unique = new HashSet<string[]>(new ArrayEqualityComparer<string>());

for (int i = list.Count - 1; i >= 0; --i)
  if (!unique.Add(list[i]))
    list.RemoveAt(i);

Thanks. Finally I get what I want to. Appreciate it! – Jacob Dec 20 '22 at 02:01 — Jacob, Dec 20 '22 at 02:01

How to check duplicate string array in list?

2 Answers2