Remove duplicates from List using Guava

Question

How can we remove duplicates from List with the help of Guava api?

Currently I am following this:

private List<T> removeDuplicate(List<T> list){
return new ArrayList<T>(new LinkedHashSet<T>(list));
}

Answer 1

Probably the most efficient way is ImmutableSet.copyOf(list).asList(), which eliminates duplicates and preserves iteration order.

(But your implementation with LinkedHashSet would be nearly as efficient, and wouldn't throw up on nulls, in the unlikely event you actually wanted nulls in your collection.)

Answer 2

I love Louis' answer for the simplicity of it (and because it's the only answer that doesn't require 2 full iterations), but unfortunately in the real world, you often encounter situations where null does occur. Here's a slightly longer null-safe version:

ImmutableSet.copyOf(
    Iterables.filter(
        list, Predicates.not(Predicates.isNull()))).asList();

Or, with static imports:

copyOf(filter(list, not(isNull()))).asList();

Of course you need to be aware of the fact that all null Values will be lost from the List.

Answer 3

with generic predicate

class DuplicateRemover<T> implements Predicate<T> {

    private final Set<T> set = new HashSet<>();

    @Override
    public boolean apply(T input) {

        boolean flag = set.contains(input);

        if (!flag) {
            set.add(input);
        }

        return !flag;
    }

}

Answer 4

If you wanna use Guava at any price you can do

return new ArrayList<T>(HashMultiSet<T>.create(list).elementSet())

Answer 5

I really don't recommend using (Linked)HashMultiSet to do task which is usually done with ArrayList and (Linked)HashSet like OP mentioned above - it's less readable for regular Java programmer and (probably) less efficient.

Instead, at least use static factory constructors like newArrayList and newLinkedHashSet to avoid all these <T>s:

private static <T> List<T> removeDuplicate(final List<T> list) {
  return Lists.newArrayList(Sets.newLinkedHashSet(list));
}

However, you can do it in more "Guava way" - by avoiding nulls and using immutable collections.

So if your collection cannot have null elements, I'd suggest using immutable set instead of mutable and less efficient one:

private static <T> List<T> removeDuplicate(final List<T> list) {
  return Lists.newArrayList(ImmutableSet.copyOf(list));
}

It's still copying objects twice, so consider being fully-immutable and changing method signature to return ImmutableList:

private static <T> ImmutableList<T> removeDuplicate(final List<T> list) {
  return ImmutableSet.copyOf(list).asList();
}

This way there is only one copying involved, because ImmutableCollection.asList() returns a view.

Answer 6

You can try Guava's MultiSet API to remove duplicates.Just add your list do the set and use the count method.

MultiSet