Read multiple files from different folders in Java

Question

What is the best way for parsing multiple files from multiple directories?

I have a folder that contains 51 sub folders and each sub folder contains 100 files.

I know how to scan one single file using

File dataFile = new File("A.txt");
    scan = new Scanner (dataFile);

    while (scan.hasNext()){ 
        System.out.print(scan.next() + "\t");
    }

but how to generalize this to read from the different directories ?

The question is quite broad. You could start by using `java.io.File` to represent the primary input, this would allow you to use `File#listFiles` to list the contents of your directory and pass each match to a method. You could also look at [Walking the File Tree](https://docs.oracle.com/javase/tutorial/essential/io/walk.html) and use it to process your files. Depending your needs, you might even consider using some kind of `ExecutorService` to allow you to process multiple files simultaneously — MadProgrammer, Jul 06 '15 at 01:11
[For example](http://stackoverflow.com/questions/12505491/list-files-from-directories-and-sub-directories-in-java-including-only-partial-f/12505570#12505570) — MadProgrammer, Jul 06 '15 at 01:13

score 0 · Answer 1 · answered Jul 06 '15 at 01:22

You should use a FileVisitor (which is an interface you implement) and then put it to work using Files.walkFileTree(Path path, FileVisitor visitor). In your case you can just subclass the SimpleFileVisitor class:

import static java.nio.file.FileVisitResult.*;

public static class PrintFiles
    extends SimpleFileVisitor<Path> {

    @Override
    public FileVisitResult visitFile(Path file,
                                   BasicFileAttributes attr) {
        if (attr.isSymbolicLink()) {
            System.out.format("Symbolic link: %s ", file);
        } else if (attr.isRegularFile()) {
            System.out.format("Regular file: %s ", file);
        } else {
            System.out.format("Other: %s ", file);
        }
        System.out.println("(" + attr.size() + "bytes)");
        return CONTINUE;
    }

}

score 0 · Answer 2 · answered Jul 06 '15 at 01:22

 void recursiveParse(String directory){
    for(File fileOrFolder : directory.listFiles()){
       if(fileOrFolder.isDirectory()){
          for (String singleFileOrFolder : fileOrFolder.listFiles()){
             recursiveParse(singleFileOrFolder); 
       }
       else{
          // parse the file
       }
    }
}

score 0 · Answer 3 · answered Jul 06 '15 at 03:19

It only works for a folder that contains subdirectories and each subdirectory doesn't contain subdirectories:

import java.io.File;
import java.io.IOException;
import java.util.Scanner;
class ScanFilesInSubFolders {
        public static void main(String[] args) throws IOException {
                System.out.println("Enter the path to search");          

                Scanner scanner = new Scanner(System.in);
                String folderPath = scanner.next();
                File [] files1 = null;
                File folder = new File(folderPath);
                if (folder.isDirectory()) {
                        File[] listOfFiles = folder.listFiles();                 
                        for (File file : listOfFiles) {
                               if(file.isDirectory()) {
           files1 = file.listFiles();  
            for(File file2 : files1)
            {
    Scanner file3 = new Scanner (file2);
    while (file3.hasNext()){ 
        System.out.print(file3.next() + "\t");
    }   
        System.out.print("\n");
            }
}
                        }
                }
        }
}

score 0 · Answer 4 · answered May 08 '17 at 14:06

Here you go :

public static void main (String[] args) throws Exception{

          ReadThroughFolders rf = new ReadThroughFolders();
            //C:\\Users\\username\\Desktop\\DOC\\COUNTRIES
          File MainDirectory = new File("C:\\Users\\username\\Desktop\\DOC\\COUNTRIES");
          rf.readDir(MainDirectory);

        }

        private void readDir(File f) throws Exception{

            Set<String>set = new TreeSet<String>();
            File []subdir = f.listFiles();
            int count = 0;
            for (File fi : subdir){

                if(fi.isFile()){

                    BufferedReader br = new BufferedReader(new FileReader(fi));
                    String line;
                    while((line = br.readLine())!= null){
                        set.add(line);                  
                    }
                    br.close();
                    count++;
                    System.out.println("reding file ----------------");
                    System.out.println(fi.getName());
                }
                String dir = f.getName();
                System.out.println("****** "+dir);
                dir = dir+".txt";
                if(subdir.length-(count) == 0){
                    System.out.println("end of directory"+dir);
                    count = 0;
                    print(set, dir);
                    set = new TreeSet<String>();
                }
                if(fi.isDirectory()){
                    //System.out.println("inside if check directory");
                    readDir(fi);
                    System.out.println("reading next directory");


                }
            }
        }

The method will read the files in the different folders in your root folder.

Read multiple files from different folders in Java

4 Answers4