How to collect the unique word and their frequency from 1 M files collection?

Question

I'm just trying to handle this problem but juggling here and there.

import java.io.*;
import java.util.*;

public class file{
  public static void main(String[] args) throws Exception { 
    Scanner in = new Scanner(System.in);
    Scanner sc=null;
    int count=0,uwords=0;
    File folder = new File("<folder path>");// The collection of files
    File[] listOfFiles = folder.listFiles();
    HashMap<String,Integer> words_fre = new HashMap<String,Integer>();

    FileWriter fw = new FileWriter("abc.txt");
    //ArrayList<String> words = new ArrayList<String>();

for (File file : listOfFiles) {

    if (file.isFile()) {
        //System.out.println(file.getName());
     try{
      sc=new Scanner(/*new BufferedReader(new File*/(file));
       //sc.useDelimiter("\\W");


       while(sc.hasNext()){
        String s = sc.next().toString();
                     s = s.replaceAll("\\<.*?>","");

                     count++; // words count   
                     if(words_fre.containsKey(s))  
                          {  
                               int a = words_fre.get(s);  
                               words_fre.put(s,a+1);             
                          }  
                     else {  
                          words_fre.put(s,1);  
                          uwords++; // unique words count   
                        }

           }

      Object[] key =   words_fre.keySet().toArray();   
          Arrays.sort(key);  
          for (int i = 0; i < key.length; i++) {  
          //System.out.println(key[i]+"= "+words_fre.get(key[i]));
          fw.write(key[i]+" : "+words_fre.get(key[i]) +"\n"); 
        }


       }catch(IOException e)  
          {  
                System.out.println(e);  
           }

  }     

  }
           /*System.out.println("Total Words = "+count);  
           System.out.println("Unique Words = "+words_fre.size());*/

           fw.write("Total Words = "+count+"\n"); 
           fw.write("Unique Words = "+words_fre.size());          
         fw.close();
 }
} 

So basically my output something like that eg.- : 3 16800 : 1 23-12-2010 : 1 7 : 1 6 : 2 8वीं : 2 अंशु : 1 अधिकतर : 2 अन्य : 1 अपने : 1 हो। : 1 ॥ : 1
: 3
I need also remove in bracket first entry [: 3 ] and second last [ ||: 1 ] and last one [:3 ]

Answer 1

Probably between your pull and push, someone else pushed some changes. To find it out, you can compare your local commit with remote commit. If there is any new commit/push after your pull, and its missing in your local, it will be root cause of the issue. You can find source of that commit on github. Below command can be used.

First find our exact remote name using command git branch -r

git log master --oneline to get local commit list

git log --oneline remote_name/master to get remote commit list

Give the list of additonal commits, which are missing at local:

git rev-list remote_name /master ^$(git rev-list master --all)