Removing duplicates from list of lists

Question

I'm having issues attempting to remove duplicate lists within a list. For example, using the list below, I need to remove one of the [2, 9] elements from the list.

List listOfLists = [[1, 5], [2, 9], [10, 12], [-1, 4], [2, 9]];

I have tried using toSet, but it only seems to work with a list of int or strings:

List uniqueLists = listOfLists.toSet().toList();

I have been stuck on this all morning, I'm sure its something simple, but the solution isn't coming to me. Any help would be appreciated.

Answer 1

The issue is that == for lists is reference equality, not based on the value of the lists.

for example if you run this program:

void main() {
  print([2, 9] == [2, 9]);
}

it will print false.

You can create a HashSet (or LinkedHashSet if you want to preserve the order) with custom equals and hashcode functions.

import 'dart:collection';
import 'package:collection/collection.dart';

void main() {
  List<List<int>> listOfLists = [
    [1, 5],
    [2, 9],
    [10, 12],
    [-1, 4],
    [2, 9],
  ];
  
  Set<List<int>> uniqueLists = HashSet<List<int>>(
    equals: const ListEquality().equals,
    hashCode: const ListEquality().hash,
  )..addAll(listOfLists);
  
  print(uniqueLists);
}

This solution uses the collection package for the ListEquality class.

Answer 2

This solution traverses the list backwards, and removes the last item when a duplicate is found. To compare lists, _listsAreEqual is used which is taken from this answer.

Less elegant compared to other solutions, but it does not require any additional packages.

bool _listsAreEqual(list1, list2) {
    var i=-1;
    return list1.every((val) {
        i++;
        if (val is List && list2[i] is List) {
            return _listsAreEqual(val,list2[i]);
        } else {
            return list2[i] == val;
        }
    });
}

List _removeDuplicates(list) {
    for (var i=list.length-1; i>0; i--) {
        for (var j=i-1; j>=0; j--) {
            if (_listsAreEqual(list[i], list[j])) {
                list.removeAt(i);
                break;
            }
        }
    }
    return list;
}

void main(){
    List listOfLists = [[1, 5], [2, 9], [10, 12], [-1, 4], [2, 9]];
    List uniqueLists = _removeDuplicates(listOfLists);
    //List uniqueLists = _removeDuplicates(List.from(listOfLists)); // leave original list unmodified
    
    print(uniqueLists);
}

If you do not want the original list to be modified, use the commented statement instead to create a copy of the original list.

Answer 3

Thank you for your help. Not sure why I didn't find this previously, but this does the trick perfectly.

List uniqueList = listOfLists.map((f) => f.toString()).toSet().toList().map((f) => json.decode(f) as List<dynamic>).toList();