find duplicate entries with streams in Java

Question

I got the following Problem:

I have to iterate over a List with students and compare their ids to find duplicates. After finding these duplicates I have to write them in a String. Sounds pretty simple so far.

students.stream()
    .filter(i-> Collections.frequency(students, i) >1)
    .forEach(System.out::println);

But when I try the code above, I don't get any duplicates printed out. I know that it isn't working because I'm not comparing the student ids, but I don't know how to compare them.

My output is blank right now.

Student looks like this:

public class Student {
    private String name;
    private int id;

    // All-args constructor, getters and setters
    // omitted for brevity
}

And this is the my example List:

public static ArrayList<Student> students = new ArrayList<>(Arrays.asList(
    new Student("Willi", 373583),
    new Student("Anselma", 476749),
    new Student("Noll", 345909),
    new Student("Inessa", 307055),
    new Student("Godart", 423496),
    new Student("Sissie", 393508),
    new Student("Allin", 434824),
    new Student("Catharine", 374286),
    new Student("Kore", 319004),
    new Student("Cornell", 325856),
    new Student("Mikkel", 468023),
    new Student("Ross", 383096),
    new Student("Robbie", 434105),
    new Student("Cariotta", 451072),
    new Student("Wendye", 334066),
    new Student("Janey", 494932),
    new Student("Nonna", 303659),
    new Student("Franklin", 460296),
    new Student("Kikelia", 466208),
    new Student("Jade", 497277),
    new Student("Traver", 451487),
    new Student("Alain", 304500),
    new Student("Jude", 335189),
    new Student("Gaile", 396638),
    new Student("Hilarius", 352284),
    new Student("Bengt", 463248),
    new Student("Brok", 473778),
    new Student("Keri", 345246),
    new Student("Ingar", 488058),
    new Student("Almeta", 422016),
    new Student("Hanny", 460693),
    new Student("Mattias", 337679),
    new Student("Cristabel", 356625),
    new Student("Banky", 320692),
    new Student("Karolina", 487674),
    new Student("Osmond", 397483),
    new Student("Essy", 384638),
    new Student("Katha", 320650),
    new Student("Dorey", 476369),
    new Student("Harlan", 499766),
    new Student("Jess", 416688),
    new Student("Bevon", 338526),
    new Student("Phaidra", 367390),
    new Student("Arthur", 341507),
    new Student("Krista", 318817),
    new Student("Riki", 470347)));

add a `map` before filter that gets the id from Student and then use that id in filter (and frequency) — Joakim Danielson, Jun 15 '20 at 17:22
Override Student `equals()` method to compare ids or add `.map(Student::id)` before filter — YuliiaZ, Jun 15 '20 at 17:25
i added .map(Student::id) but im not sure what to write into the filter, thank you for your help — Pody, Jun 15 '20 at 17:53
Can you [edit] your question and post the definition of your `Student` class? Also can you post a sample `Student` list and the output you want to get from that sample list? — Abra, Jun 15 '20 at 17:59
The only thing you didn't add is the output you want to get after processing your sample list. — Abra, Jun 15 '20 at 19:45

score 2 · Answer 1 · answered Jun 15 '20 at 17:29

2

For each element you have to traverse students to check the frequency. A bit more optimized version would be to group it once and print the ones, that are duplicated:

students.stream()
    .collect(Collectors.groupingBy(Function.identity(), Collectors.counting()))
    .entrySet().stream()
    .filter(e -> e.getValue() > 1)
    .map(Map.Entry::getKey)
    .forEach(System.out::println);

answered Jun 15 '20 at 17:29

Andronicus

25,419
17
47
88

1

Ok So I tried that, but it still didn´t print out anything. after I set the filter value to 0 it prints out some references. of the students objects but not the IDs – Pody Jun 15 '20 at 17:43
@Pody you need to override `equals` and `hashCode` – Andronicus Jun 15 '20 at 17:52
Thank you for your help, I'm pretty new to Java and Lamdas but could you show me in which way I have to override equals and hashCode? – Pody Jun 15 '20 at 17:56
@Pody you don't have to do that manually, your ide will take care of that. Foe example if you're using intellij, here's how to do that (section "euqals and hashCode"): https://www.jetbrains.com/help/idea/generating-code.html – Andronicus Jun 15 '20 at 18:00
1

@Pody `Collectors.groupingBy(Student::getId, ...` would be an appropriate usage for printing just the ids for those which have multiple occurrences in this existing code. – Naman Jun 16 '20 at 00:36
1

@Pody This is a good approach, but replace `Function.identity()` with `Student::getId` as Naman suggests. – erickson Jun 16 '20 at 15:12

score 1 · Answer 2 · answered Jun 15 '20 at 17:34

1

One of the possible solutions would be to implement a stateful Predicate:

public static <T> Predicate<T> distinctByKey(
    Function<? super T, ?> keyExtractor) {

    Map<Object, Boolean> seen = new ConcurrentHashMap<>(); 
    return t -> seen.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) == null; 
}

And to get a new filtered collection by Id, you can use:

List<Student> studentListFiltered = studentList.stream() 
  .filter(distinctByKey(s -> s.getId())) 
  .collect(Collectors.toList());

answered Jun 15 '20 at 17:34

aballaci

1,043
8
19

Worth linking some source - [Java 8 Distinct by property](https://stackoverflow.com/questions/23699371/java-8-distinct-by-property). But this collects distinct `Student` and not print the ids for those which are duplicates as in the question. – Naman Jun 16 '20 at 00:37

find duplicate entries with streams in Java

2 Answers2