Iterating 2 files to remove any common string lines and separate out unique strings

Question

public static void main(String[] args) throws IOException {



    FileReader a = new FileReader(new File("C:\\Users\\IBM_ADMIN\\Desktop\\SAM_PUBLIC_MONTHLY_20150802\\a.txt"));  // new file

    FileReader b = new FileReader(new File("C:\\Users\\IBM_ADMIN\\Desktop\\SAM_PUBLIC_MONTHLY_20150802\\b.txt"));  // new file

    @SuppressWarnings("resource")
    BufferedReader file1 =new BufferedReader(a); //new

    @SuppressWarnings("resource")
    BufferedReader file2 =new BufferedReader(b); //old
    PrintWriter  Uniquefile = new PrintWriter (new File("C:\\Users\\IBM_ADMIN\\Desktop\\SAM_PUBLIC_MONTHLY_20150802\\samnew.dat"));

    List<String> list= new ArrayList<String>();
    while(file1.readLine()!=null)
    {
        String str=file1.toString();
        list.add(str);
        while(file2.readLine()!=null)
        {
            String str1=file2.toString();
            if(list.contains(str1))
            {
                list.remove(str);
            }
        }
    }

    Iterator<String> it=list.iterator();
    while(it.hasNext())
    {
        String str2=it.toString();
        Uniquefile.write(str2);
    }
}

I am Iterating 2 files to remove any common string lines and separate out unique strings.

Ex: File 1 .txt has string lines "1 2 3" and file 2.txt has "2 3" , I want to print "1" in File 3.txt.

Hope my question is clear. Would be great if someone can correct my code shown below. Thanks

@ScottMcGready check if any of the answers fit your needs, and ask for clarification if needed..... — Jordi Castilla, Sep 07 '15 at 14:54

score 2 · Answer 1 · answered Sep 07 '15 at 14:49

If I understood you correctly, you want to get all distinct lines from two files. I recommend using Set Collection

to show how it works here is snippet of the code:

Set<String> set1 = new HashSet<String>();
Set<String> set2 = new HashSet<String>();

String line=null;

while((line=file1.readLine())!=null){
  set1.add(line);
}

while((line=file2.readLine())!=null){
  set2.add(line);
}


//get similars
Set<String> similars=new HashSet<String>(set1);

similars.retainAll(set2);

set1.removeAll(similars);
set2.removeAll(similars);

//all distinct lines
Set<String> distinctSet=new HashSet<String>(set1);
disctinctSet.addAll(set2);

score 0 · Answer 2 · answered Sep 07 '15 at 14:53

If files are not huge, my approach will be:

Read both files
Store old values in a List
Iterate over new values
Write non repeated values to new file

public static void main(String[] args) throws Exception {
    // GET BOTH FILES, BUFFERS AND READERS
    String pathname = "C:\\tmp\\";
    FileReader f1 = new FileReader(new File(pathname, "1.txt"));
    FileReader f2 = new FileReader(new File(pathname, "2.txt"));
    BufferedReader b1 = new BufferedReader(f1);
    BufferedReader b2 = new BufferedReader(f2);

        FileWriter writer = new FileWriter("C:\\Users\\IBM_ADMIN\\Desktop\\SAM_PUBLIC_MONTHLY_20150802\\samnew.dat");

    // STORE OLD VALUES INTO A LIST
    List<String> old = new ArrayList<>();
    String l2 = b2.readLine();
    while (l2 != null) {
        old.add(l2);
        l2 = b2.readLine();
    }

    // ITERATE OVER NEW VALUES
    String line = b1.readLine();
    while (line != null) {
        // IF NOT CONTAINED IN OLD VALUES, PRINT OR WRITE IN FILE
        if (!old.contains(line)) {
            writer.write(line);
            writer.write(System.lineSeparator());

            System.out.println(line);
        }
        line = b1.readLine();
    }

    b1.close();
    b2.close();
}

ur only checking if file2 doesn't contain a string that is in file1, not vice versa .( then again it might be what @Suraj wants as the question is abit vague) — nafas, Sep 07 '15 at 15:05

Iterating 2 files to remove any common string lines and separate out unique strings

2 Answers2