remove duplicate list from an arrayList using Set

Question

I have a List that contains duplicate ArrayList.

I'm looking for a solution to remove them.

Here is an example:

listOne = [[1, 0], [0, 1], [3, 2], [2, 3]]

This set contains duplicate List. Normally i want to get :

theListAfterTransformation  = [[1, 0],[3, 2]]

Here is my tiny example, i tried to use the Set but it didn't work well.

public class Example {
    public static void main( String[] args ) {
        ArrayList<ArrayList<Integer>> lists = new ArrayList<>();

        ArrayList<Integer> list1 = new ArrayList<>(); list1.add(1); list1.add(0);
        ArrayList<Integer> list2 = new ArrayList<>(); list2.add(0); list2.add(1);
        ArrayList<Integer> list3 = new ArrayList<>(); list3.add(3); list3.add(2);
        ArrayList<Integer> list4 = new ArrayList<>(); list4.add(2); list4.add(3);

        lists.add(list1);lists.add(list2);lists.add(list3);lists.add(list4);

        System.out.println(getUnduplicateList(lists));


    }
    public static ArrayList<ArrayList<Integer>> getUnduplicateList( ArrayList<ArrayList<Integer>> lists) {
        Iterator iterator = lists.iterator();
        Set<ArrayList<Integer>> set = new HashSet<>();
        while (iterator.hasNext()){
            ArrayList<Integer> list = (ArrayList<Integer>) iterator.next();
            set.add(list);
        }
        return new ArrayList<>(set);
    }

}

Note that is a tiny example from my project and it will be very hard to use a solution that change many thing in this implementation.

So take into account that the getUnduplicateList should keep the same signature. the good idea will be to change only the implementation.

This program print the same list as the input. any idea please.

Answer 1

A couple notes on terminology—Set is a distinct data structure from List, where the former is unordered and does not allow duplicates, while the latter is a basic, linear collection, that's generally ordered, and allows duplicates. You seem to be using the terms interchangeably, which may be part of the issue you're having: Set is probably the appropriate data structure here.

That said, it seems that your code is relying on the List API, so we can follow that along. Note that you should, in general, code to the interface (List), rather than the specific class (ArrayList).

Additionally, consider using the Arrays.asList shorthand method for initializing a list (note that this returns an immutable list).

Finally, note that a HashSet eliminates duplicates by checking if both objects have the same hashCode. Lists containing the same elements are still not considered to be the same list unless the elements appear in the same order, and will typically not be treated as duplicates. Sets, however, implement equals and hashCode in such a way that two sets containing exactly the same elements are considered equal (order doesn't matter).

---

Using your original starting collection, you can convert each inner-list to a set. Then, eliminate duplicates from the outer collection. Finally, convert the inner-collections back to lists, to maintain compatibility with the rest of your code (if needed). This approach will work regardless of the size of the inner-lists.

You can simulate these steps using a Stream, and using method references to convert to and from the Set, as below.

---

import java.util.List;
import java.util.Arrays;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.stream.Collectors;

public class Example {

    public static void main( String[] args ) {
        List<Integer> list1 = Arrays.asList(1, 0);
        List<Integer> list2 = Arrays.asList(0, 1);
        List<Integer> list3 = Arrays.asList(3, 2);
        List<Integer> list4 = Arrays.asList(2, 3);

        List<List<Integer>> lists = Arrays.asList(list1, list2, list3, list4);

        System.out.println(getUnduplicateList(lists));
    }

    public static List<List<Integer>> getUnduplicateList(List<List<Integer>> lists) {
        return lists
                .stream()
                .map(HashSet::new)
                .distinct()
                .map(ArrayList::new)
                .collect(Collectors.toList());
    }
}

Answer 2

You need to convert the inner lists to sets as well.

Answer 3

Another solution is to sort your lists and then run them through distinct Although this is not very efficient and you will also obtain a set of sorted lists:

Set<List<Integer>> collect = set.stream()
            .map(list -> {
                list.sort(Comparator.comparingInt(Integer::intValue));
                return list;
            })
            .distinct()
            .collect(Collectors.toSet());