Remove ArrayList duplicates in a text file - Java

Question

I want to remove the duplicates of userNames in the ArrayList. I've tried to convert the ArrayList to an HashSet, but for some reason it doesn't work. The reason why I chose to convert the ArrayList into a HashSet is because it does not allow duplicated values, however, when I use it on my code, it only changes the order in the list:

My code output:

Choreography - Imran Sullivan
Goodfella - Khalil Person
DarknianIdeal - Sophia Jeffery
DarknianIdeal - Clyde Calderon
Frolicwas - Taylor Vargas
Reliable - Aryan Hess
DarknianIdeal - Liyah Navarro
Deservedness - Eadie Jefferson
Reliable - Angel Whitehouse
Choreography - Priya Oliver

How the output should be:

Choreography - Imran Sullivan
Goodfella - Khalil Person
DarknianIdeal - Sophia Jeffery
Frolicwas - Taylor Vargas
Reliable - Aryan Hess
Deservedness - Eadie Jefferson

This is the code. I've splitted the data into an Array so I can print out the data individually.

import java.util.*;
import java.io.*;

public class Task1 {
    public static void main(String[] args) {   
        List<Person> personFile = new ArrayList<>();
        Set<Person> splitUserNameList = new HashSet<>(personFile);

        try {            
            BufferedReader br = new BufferedReader(new FileReader("person-data.txt"));
            String fileRead = br.readLine();
            while (fileRead != null) {
                String[] personData = fileRead.split(":");                
                String personName = personData[0];
                String userNameGenerator = personData[1];
                String[] userNameSplit = userNameGenerator.split(",");                
                String newUserNameSplit = userNameSplit[0];                
                Person personObj = new Person(personName, newUserNameSplit);
                splitUserNameList.add(personObj);                
                fileRead = br.readLine();
            }
            br.close();            
        }  
        catch (FileNotFoundException ex) {            
            System.out.println("File not found!");
        } 
        catch (IOException ex) {             
            System.out.println("An error has occured: " + ex.getMessage());
        }

        for (Person userNames: splitUserNameList) {                
            System.out.println(userNames);           
        }           
    } 
}

/* Person Class */

public class Person {
    private String personName;
    private String userNameGenerator;

    public Person(String personName, String userNameGenerator) {
        this.personName = personName;
        this.userNameGenerator = userNameGenerator;
    }

    public String getPersonName() {
        return personName;
    }

    public String getUserNameGenerator() {
        return userNameGenerator;
    }

    @Override
    public String toString() {        
        return userNameGenerator + " - " + personName;
    }
}

Perhaps the problem comes from the fact that Person does not override equals and hashcode(), so the Set can't know which Person are supposed to be equals — baudsp, Dec 05 '17 at 16:22
@rossum that's the output of `System.out.println`, not the file. Those `-` are printed — Oneiros, Dec 05 '17 at 16:23
@User786 It's true, `HashSet` doesn't allow duplicate elements but in order to recognize a duplicate you have to provide a way to test the **equality** of two `Person`. From the `HashSet` point of view, `Reliable - Aryan Hess` and `Reliable - Angel Whitehouse` are two different objects, not duplicates. As Michael says, you have to implement the `equals()` and `hashCode()` methods — Oneiros, Dec 05 '17 at 16:26

Michael · Accepted Answer · 2017-12-05T16:30:26.597

You need to override the equals and hashCode methods on your Person object in order for Set to know which objects are considered the same.

It seems you want any two people with the same userNameGenerator field to be considered equal. In that case, the following will suit your needs:

public class Person {
    private String commonName;
    private String userNameGenerator;

    ...

    @Override
    public int hashCode() 
    {
        return userNameGenerator.hashCode();
    }

    @Override
    public boolean equals(Object o) 
    {
        if (this == o) return true;

        if (this.getClass() != o.getClass()) return false;

        final Person that = (Person) o;

        return this.userNameGenerator.equals(that.userNameGenerator);
    } 
}

Some important things to note:

In the case of duplicates, the Set will only allow in the first one so insertion order becomes important.
Sets should not contain mutable elements because mutability destroys their internal consistency. Your Person class is immutable (has no setters) which is good but you may want to enforce this immutability even further by declaring all of it's fields final.

Note for OP, to not lose much of his time on searching how to implement equals and hashCode, most IDE's have option for generating code automatically for equals and hashCode, just search for it in your IDE documentation. — whatamidoingwithmylife, Dec 05 '17 at 16:25
`(o instanceof Person)` leads to potential wrong (asymmetric) `equals` implementations when it comes to subclassing. Better use `o.getClass() != getClass()`. — Florian Albrecht, Dec 05 '17 at 16:29

Remove ArrayList duplicates in a text file - Java

1 Answers1