Read and discard data CSV

Question

I have one csv with one row who have diferent users (users.csv), in the other hand I also have a csv with users (users2.csv).. The problem is that I want to "compare?" these two documents and discard users from users2.csv to users1.csv if they exist in this file. Please ideas or advice, how could I do it??

What is the exact difficulty you face when solving this problem? — niyasc, Aug 20 '19 at 08:43
Create a User bean, read both files in to list of beans with http://opencsv.sourceforge.net or any other library, compare two lists. I would make a map from each list by user name/e-mail/whatever, and compare if left map keys exists in right map keys. Should take not more than an hour to do that. — Dmitri Algazin, Aug 20 '19 at 08:46

score 2 · Answer 1 · answered Aug 20 '19 at 09:27

Load the first file into a List<String> users.
Load the second file into a List<String> users2.
use apache commons-collections CollectionUtils.removeAll(Collection<E> users, Collection<?> users2)

To load a file in a list you can find inspiration here.

Et voilà.

This only works if the size of the files is acceptable to load in memory. Otherwise it requires another approach like sorting both files using command line sort commands and walk through both files reading line by line and decide to write to output or not.

score 1 · Answer 2 · answered Aug 20 '19 at 09:11

1

You can use BeyondCompare to compare the two csvs. It will distinctively identify the missing user along with other data mismatch if any. In case if you want to do it programatically, you can create a user bean (and override equals method to compare username or any other you want) after copying csv into list/map of beans.

answered Aug 20 '19 at 09:11

Bala Ji

35
6

I have to copy the two rows of a different csv into a map and then compare it to UserBean by creating a new document that only contains the useful information?. – Lewis Aug 20 '19 at 09:28

score 1 · Answer 3 · answered Aug 20 '19 at 09:35

Best way I see,

1) Read both the files using Java NIO Api (That's actually very fast)separately and store them into list.

    Path path = Paths.get("src/main/resources/shakespeare.txt");
    try {

      Files.lines(path).forEach(System.out::println);//print each line

    } catch (IOException ex) {
      ex.printStackTrace();//handle exception here
    }

2) Compare both list using java 8 predictor.

    public static List < String > filterAndGetEmployees(List < String> employees,
        Predicate < String > predicate) {
        return list.stream().filter(predicate).collect(Collectors. < String > toList());
    }

3) If you wish to write file again , You can go like,

    Path path = Paths.get("src/main/resources/shakespeare.txt");
    try(BufferedWriter writer = Files.newBufferedWriter(path, Charset.forName("UTF-8"))){
            writer.write("To be, or not to be. That is the question.");
    }catch(IOException ex){
            ex.printStackTrace();
    }

Hope this will help you..

Read and discard data CSV

3 Answers3