Removing duplicate IEnumerable from IEnumerable>

Question

How to remove duplicate content from a given collection of collections? not taking order into account.

for eg.

IEnumerable<IEnumerable<T>> Collection = [{1,2,3} , {2,1,3}, {1,1,2}, {1,2,2}]

should be

IEnumerable<IEnumerable<T>> Collection = [{1,2,3} , {1,1,2}, {1,2,2}] // [ {2,1,3}, {1,1,2}, {1,2,2}] is also valid

** Edit **

I understand, for IEnumerables E1 and E2 I could do something like this to find the duplicates:

bool equal = (E1.Count() == E2.Count() && (!E1.Except(E2).Any() || !E2.Except(E1).Any()))

if(equal)
{
    //Do something
}

However, how to do something similar for

IEnumerable<IEnumerable<T>>

Have you tried anything already? What are your ideas of solving this? This is not a code writing service. SO expects you to show some effort before you come and ask a specific question. — Maciej Jureczko, Aug 30 '17 at 06:34
Please provide some code that compiles and show your previous efforts. — Sefe, Aug 30 '17 at 06:52
You can find this thread quite useful: https://softwareengineering.stackexchange.com/questions/339317/find-duplicate-in-a-list-of-a-list-of-integers/339318#339318 — Absurd, Aug 30 '17 at 07:02

score 1 · Accepted Answer · answered Aug 30 '17 at 07:51

The trick is to build a new list, containing only unique items. Your check does not work for all edge cases (e.q your check is successful for {1, 1, 2} == {1, 2, 3}, because value types can only be compared for equality, not identity).

The easiest way is to sort the collection and than use Enumerable.SequenceEqual

public static class Helper
{
   public static IEnumerable<IEnumerable<int>> Unique(this IEnumerable<IEnumerable<int>> source)
   {
      var list = new List<List<int>>(); // sorted reference list.

      foreach (var toCompare in source)
      {
         var toComp = toCompare.OrderBy(x => x).ToList(); // prevent multiple enumerations.
         if (!list.Any(item => toComp.SequenceEqual(item)))
         {
            list.Add(toComp);
            yield return toCompare; // return the unsorted one!
         }
      }
   }
}

The usage

var unique = collection.Unique();

How does the code above work:

We keep a reference list of all returned items, but the reference list is sorted. Then we enumerate the source list, sort every item and check if it is already in our reference list, if not we add the sorted item to the reference list and yield return the unsorted original item.

Removing duplicate IEnumerable from IEnumerable>

1 Answers1