Passing multiple text files in java for the purpose of parsing

Question

I am trying to build a parser which would take up each file from a given path which contains 100's of text files , parse each and save each files extracted data into a different document in memory. My doubt is how do I specify to Java to pick up each text file from the given location and parse it. I want to kind of construct a loop that would check through all the folders inside the given path pick up a text file process it and move to the next location.

Answer 1

I suggest that you break down this task into steps. Describe each step in English. Note that when you start a project like this, you should mostly ignore the programming language you are going to use. For example in this case, the algorithm might look something like this:

Get directory name from user
For each file in the directory
    Open the file and parse it

When you write out a description like this, you often will get steps that need to be broken down further. These can (and should) be implemented as functions. Get directory name from user is such a step in this example, as is Open the file and parse it.

You should repeat the process for each of these steps: describe in words how to accomplish the task. Eventually you will reach a level where the words are nearly identical to programming language constructs. When you get to this point, you can begin translating your description into Java or any other appropriate programming language.

If, at any point in this process, you have problems, you can come back to SO and post a specific question including what you have tried up to that point. Also be sure to write about the exact problem encountered. You should include any relevant error message as well as describe what you expect to happen.

Answer 2

File filesDirectory = new File(directory);
File[] files; //holds files in given directory
files = filesDirectory.listFiles();
for(File file:files)
{
//Your code here parse contents of each file individually
//Ex.
     if(!file.isDirectory())
     {
        String fileName = null;
        fileName = file.getPath();
        if(fileName.indexOf(".txt")
        {
          //read contents of each file and store in memory
        }
     }
}

Answer 3

First, get a list of the files in the directory:

File directory = ...;
File[] files = directory.listFiles();

Now you can do something with each file:

for (File file : files) {
    // do something
}

Answer 4

Using filevisitor will be efficient here, A facility provided by java.nio package Here I have written a small code for you. Check out :

import static java.nio.file.FileVisitResult.*;

public static class ParseFiles extends SimpleFileVisitor<Path> {

    @Override
    public FileVisitResult visitFile(Path file, BasicFileAttributes attr) {
        if(give your condition if you want to parse file or not) {
            parseFile(file) //I assumed the function to be used to parse file
        }
        return CONTINUE;  // It will make it to recursively search for file
    }
}

Now you will only need to start parsing for all files in directory using Files.walkFileTree.

ParseFiles pf = new ParseFiles();
Files.walkFileTree(basedir, pf); // basedir is String

for Reference : docs .