How to remove duplicate elements from list?

Question

List list = ["f1,f2","f2,f3","f4,f5","f2,f1","f5,f4"];

output list would be List uniqueList = ["f1,f2","f2,f3","f4,f5"]

@shmosel Not that simple. `f2,f1` and `f1,f2` are being considered as logically equivalent. — Tim Biegeleisen, Jul 03 '19 at 01:24
How about this: `list.stream().map(s -> Set.of(s.split(","))).distinct().map(s -> String.join(",", s)).collect(Collectors.toList())` — shmosel, Jul 03 '19 at 01:35
i am getting comile time error- Cannot infer type argument(s) for map(Function super T,? extends R>) — karthik muthyam, Jul 03 '19 at 02:20
It should be able to do this using a set with a custom comparator. — Tim Biegeleisen, Jul 03 '19 at 10:35
@Holger Possibly. But it's not clear if that's a possible value or what the correct representation would be. — shmosel, Jul 03 '19 at 16:25

score 1 · Answer 1 · answered Jul 03 '19 at 05:41

Using an additional class:

    static class Pair {
    String a, b;

    Pair(String s) {
        String[] arr = s.split(",");
        this.a = arr[0];
        this.b = arr[1];
    }

    static String pairToString(Pair p) {
        return p.a + "," + p.b;
    }

    @Override
    public int hashCode() {
        return Objects.hash(a, b) + Objects.hash(b, a);
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;
        Pair p = (Pair) o;
        return (p.a.equals(a) && p.b.equals(b)) || (p.b.equals(a) && p.a.equals(b));
    }

}

Now you can use:

 public static void main(String[] args) {
        List<String> list = Arrays.asList("f1,f2", "f2,f3", "f4,f5", "f2,f1", "f5,f4");
        List<String> strings = list
                .stream()
                .map(Pair::new)
                .distinct()
                .map(Pair::pairToString)
                .collect(Collectors.toList());
    }

Piotr Durlak · Accepted Answer · 2019-07-03T18:42:18.337

I have another solution . If you dont want to prepare another class to compare values inside List . You can separete each value by comma and sort those data. After that you can again converte them to Set of String

public static void main(String[] args) {
    List<String> stringList = Arrays.asList("f1,f2", "f2,f3", "f4,f5", "f2,f1", "f5,f4");
    Set<String> result = new HashSet<>();
    for (String s : stringList) {
        String[] elements = s.split(",");
        Arrays.sort(elements);
        result.add(Arrays.toString(elements));
    }

    for (String e : result){
        System.out.println(e);
    }
}

score 0 · Answer 3 · answered Jul 03 '19 at 04:05

I have created a class to model the pairs and override the equals method to treat "f1,f2" and "f2,f1" as equals and then found out the duplicates using HashSet.

import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;

public class so1 {
    public static void main(String[] args) {

        List<String> list = Arrays.asList(new String[] {"f1,f2","f2,f3","f4,f5","f2,f1","f5,f4"});

        List<pair> plist = new ArrayList<pair>();
        for (int i = 0; i < list.size(); i++) {
            plist.add(new pair(list.get(i)));
        }

        HashSet<pair> hs = new HashSet<pair>(); 
        for (int i = 0; i < plist.size(); i++) {
            if(!hs.add(plist.get(i))){
                System.out.println("Found duplicate "+plist.get(i).toString());
            }
        }

        List<String> uniqueItems = new ArrayList<String>();
        for (Iterator iterator = hs.iterator(); iterator.hasNext();) {
            pair pair = (pair) iterator.next();
            uniqueItems.add(pair.toString());
        }

        System.out.println(uniqueItems);


    }
}

class pair{
    pair(String inp){
        String[] tokens = inp.split(",");
        Arrays.sort(tokens);
        for(String t: tokens){
            elements.add(t);
        }
    }
    List<String> elements = new ArrayList<>();

    @Override
    public String toString() {
        return ""+elements;
    }

    @Override
    public int hashCode() {
        final int prime = 31;
        int result = 1;
        result = prime * result + ((elements == null) ? 0 : elements.hashCode());
        return result;
    }

    @Override
    public boolean equals(Object obj) {
        if (this == obj)
            return true;
        if (obj == null)
            return false;
        if (getClass() != obj.getClass())
            return false;
        pair other = (pair) obj;
        if (elements == null) {
            if (other.elements != null)
                return false;
        } else if (!elements.equals(other.elements))
            return false;
        return true;
    }
}

user11645321 · Answer 4 · 2019-07-03T01:30:16.027

-6

Here are a couple answers from https://www.geeksforgeeks.org/how-to-remove-duplicates-from-arraylist-in-java/

Java 7

    // Create a new ArrayList 
    ArrayList<String> uniqueList = new ArrayList<String>(); 

    // Traverse through the first list 
    for ( element : list) { 

        // If this element is not present in uniqueList
        // then add it 
        if (!uniqueList.contains(element)) { 

            uniqueList.add(element); 
        } 
    }

Java 8

    List<String> uniqueList = list.stream() 
        .distinct() 
        .collect(Collectors.toList());

edited Jul 03 '19 at 01:30

answered Jul 03 '19 at 01:29

user11645321

11
2

https://stackoverflow.com/questions/56861754/how-to-remove-duplicate-elements-from-liststring?noredirect=1#comment100273074_56861754 – shmosel Jul 03 '19 at 01:29
3

your solution is giving the same input as an output – karthik muthyam Jul 03 '19 at 01:39
1

f1,f2 and f2,f1 are not logically equal – karthik muthyam Jul 03 '19 at 02:21

How to remove duplicate elements from list?

4 Answers4