How to read data from multiple .csv files with similar names?

Question

I have multiple files that start as CUSTOMER_YYYYMMDD.csv (each file has its own date) in a folder that consists of many other files. My script (that uses BufferedReader) already can read data from one concrete file, but I want to get data from files only named as CUSTOMER_YYYYMMDD.csv, but there is nothing on my mind how to do that.

Here's what I've got:

public static void main(String[] args) 
{

    BufferedReader br = null;

    try {

        String line;

        br = new BufferedReader(new FileReader("/Users/ovshievb/Desktop/IP/data/tcos/INPUT/CUSTOMER_20150401.csv"));

        //Nacita hlavicku CSV failu aby ji preskocit
        br.readLine();

        // Cteni failu radek po radku
        while ((line = br.readLine()) != null) {
        //  System.out.println("Raw CSV data: " + line);
            System.out.println("Customer: " + csvToArray(line) + "\n");
        }

    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        try {
            if (br != null) br.close();
        } catch (IOException exception) {
            exception.printStackTrace();
        }
    }
}

the exact same way you read any other file. Just because they have similar names on the disk, doesn't mean they need to have identical names for references in your code. — Stultuske, Sep 08 '15 at 13:39
You need to filter first the filenames. Maybe this SO answer could help. http://stackoverflow.com/questions/29294470/get-filename-with-date-and-trailing-counter#29296475 — SubOptimal, Sep 08 '15 at 13:50
@SaurabhJhunjhunwala I want somehow get data from all files that start with CUSTOMER_********. — B. Ovshiev, Sep 08 '15 at 13:54
have a look at http://stackoverflow.com/questions/1844688/read-all-files-in-a-folder. you have a similar problem — Saurabh Jhunjhunwala, Sep 08 '15 at 13:57

score 2 · Accepted Answer · answered Sep 08 '15 at 13:46

2

At first you should get a list of all files that are in your directory. This can be done by createing a File Object of your directory and then call the list() method.

File myDirectory = new File(/path/to/my/dir/);
String[] containingFileNames = myDirectory.list();

After that you can iterate over all file names and check if the name matches the pattern CUSTOMER_YYYYMMDD.csv

for (String fileName : containingFileNames) {
   if (fileName.matches("CUSTOMER_\\d*.csv") {
      // Read this file with the BufferedReader like in your code above
   }
}

answered Sep 08 '15 at 13:46

Sebastian

1,642
13
26

correct answer but check FileNameFilter, adding to `accept` to class `if (fileName.matches("CUSTOMER_\\d*.csv") return true;` will do same much more clean ;) – Jordi Castilla Sep 08 '15 at 13:48
Or even cleaner, `return (fileName.matches("CUSTOMER_\\d*.csv");`, if you wanted to go that way. – edwardmlyte Sep 08 '15 at 13:59
fileName.matches("CUSTOMER_\\d*.csv") will return true for a file named CUSTOMER_999999999999.csv which doesn't respect the requirement of the last part being a date. – pmartin8 Sep 08 '15 at 15:35
Since the OP wanted a solution for files that "start with CUSTOMER_" this fits the requirement. If the check should include a correct date format tough, the OP might take a look into [Regular Expressions](http://www.vogella.com/tutorials/JavaRegularExpressions/article.html) – Sebastian Sep 09 '15 at 06:28
If the requirement is to "start with CUSTOMER_", using \\d* is useless in your solution. Also @Schlangguru make sure you add a comment on a solution when you vote it down. :) – pmartin8 Sep 10 '15 at 18:29
Yep, thats true. "CUSTOMER_.*" would do the trick. – Sebastian Sep 10 '15 at 19:53

score 1 · Answer 2 · answered Sep 08 '15 at 13:45

1

You can get a filtered list of files in a folder/directory using File#listFiles(FilenameFilter). Just provide an implementation for the filter - you can check the filename against a regular expression.

answered Sep 08 '15 at 13:45

Vlad

10,602
2
36
38

score 0 · Answer 3 · answered Sep 08 '15 at 13:43

0

make your block of code above into a method that takes in one parameter for the file name to read. Then all you have to do is call your method for any other file you want to read.

answered Sep 08 '15 at 13:43

scottDeveloper

159
2
17

score 0 · Answer 4 · answered Sep 08 '15 at 13:47

Fetch all the CSV files in the folder first. And then read them recursively. Sample code to fetch all CSV files in a folder is given below.

 public static List<String> parseForCsvFiles(String parentDirectory){
    File[] filesInDirectory = new File(parentDirectory).listFiles();
    List<String> fileList = new ArrayList<String>();
    for(File f : filesInDirectory){
        if(f.isDirectory()){
            parseForCsvFiles(f.getAbsolutePath());
        }
        String filePath = f.getAbsolutePath();
        String fileExtenstion = filePath.substring(filePath.lastIndexOf(".") + 1,filePath.length());
        if("csv".equals(fileExtenstion)){
            //add to a list or array
            fileList.add(filePath);
        }

    }       
  return fileList;
}

pmartin8 · Answer 5 · 2015-09-08T15:24:44.387

Try this:

SimpleDateFormat df = new SimpleDateFormat("yyyyMMdd");
    File folder = new File("/Users/ovshievb/Desktop/IP/data/tcos/INPUT/");
    for(File file: folder.listFiles()){
        String name = file.getName();
        if(name.startsWith("CUSTOMER_")){
            String date = name.replace("CUSTOMER_", "");
            BufferedReader br = null;
            try{
                String line;

                df.parse(date);
                br = new BufferedReader(new FileReader("/Users/ovshievb/Desktop/IP/data/tcos/INPUT/"+ name));

                //Nacita hlavicku CSV failu aby ji preskocit
                br.readLine();

                // Cteni failu radek po radku
                while ((line = br.readLine()) != null) {
                //  System.out.println("Raw CSV data: " + line);
                    System.out.println("Customer: " + csvToArray(line) + "\n");
                }
            }catch(ParseException ex){
                // Wrong date format.
            } finally {
                br.close();
            }
        }
    }

I think you meant `String name = file.getName();`. Plus let's not encourage new devs into catching generic exceptions. — edwardmlyte, Sep 08 '15 at 13:58

How to read data from multiple .csv files with similar names?

5 Answers5