Java: Get a list of new files from a directory

Question

I need a java program that on demands returns a list of newly added files to a directory. My definition of new file is the one which was not there at last checkup/scanning of the directory.

I know there are libraries such as JNotify, and java directory watch service http://docs.oracle.com/javase/tutorial/essential/io/notification.html. However in my understanding these libraries implement listeners that watch a directory and notify as soon as some file event (add, remove, delete, etc) occurs. But what I need is a method which on demand returns a list of newly added files to a directory.

Newly added compared to what? Created since the last time it was ran? What exactly do you mean by "newly added"? — David, Mar 18 '13 at 11:14
@david99world as I said "newly added is the one which was not there at last call i.e. checkup/scanning of the directory — Nadeem, Mar 18 '13 at 13:03

score 1 · Answer 1 · answered Mar 18 '13 at 11:19

1

You can achieve this easily by maintaining last request timestamp with the caller. Then pass that timestamp to a java class which will scan the directory for files created after that timestamp.

Something like:

public List<File> getNewFiles(Date lastRequestedDate)
{
//Iterate the directory for files newer than the date passed in as parameter
}

answered Mar 18 '13 at 11:19

Manish

3,913
2
29
45

Manish thanks for the idea, actually I was looking for a link to some already done work, and reuse it for my own. Ad reuse is a good practice. But if I dont get it in good time, then the search process may take longer than what it saves. :) – Nadeem Mar 18 '13 at 12:58

score 1 · Answer 2 · edited May 23 '17 at 12:21

There are different approaches.

Approach one: File creation date

You might want to list all files and check the creation timestamp of each file. There's already a deep discussion on this topic. Just check if the timestamp is larger than the timestamp you saved when you last checked the directory.

Approach two: Create and maintain a list of files

If you can afford the memory and the time, create a list of files in the directory. Iterate over the directory and add all the files that are not already in the list to another list. Do whatever you need to do with them and add them to your main list.

You should also serialize your main list and save it to the disk after every successful checking and working with files.

score 0 · Answer 3 · answered Mar 18 '13 at 11:16

0

You can maintain a list of file names and check against that for new files

answered Mar 18 '13 at 11:16

Abdullah Shaikh

2,567
6
30
43

2

No need to maintain list of files. You could just store the timestamp when last request was made and then return the list of files newer than the timestamp. – Manish Mar 18 '13 at 11:18

score 0 · Answer 4 · answered Mar 20 '13 at 16:31

I would suggest that you go with Java directory watch service. It should be fairly quick to implement a watch service. Have it running in the background; it will keep queueing events - add, remove, delete - as they happen. Whenever your on-demand API needs the list of files that have been added, you just need to do the following:

poll all the events that have been queued so far
iterate over them, look for the add event types and create the list to be returned
clear the events that you've polled so that they don't appear in the next poll (there is a reset API to do this)

The benefits of this approach would be:

Fewer issues: You would have to scan directory and keep track of file being added; Java would be doing that for you, so there won't be any bugs in that part.
Less memory consumption: With the timestamp approach, you will get files that have been modified since the last scan, so you would have to keep a list of all files you have encountered to identify file that have been added. But with the directory watch service you would have have to maintain any such data structure.
Better performance: During the on-demand method execution, you would not be performing directory listing or file name comparisons; you would just iterate over the already queued events and create the return value.

score 0 · Answer 5 · answered Jun 11 '14 at 09:23

You can maintain a file pointer in separate files.. and every time when you need new/older file you can pass into below method.

package test;

import java.io.File;
import java.io.FileFilter;
import java.io.IOException;
import java.util.Arrays;
import java.util.Date;

import org.apache.commons.io.comparator.LastModifiedFileComparator;
import org.apache.commons.io.filefilter.AgeFileFilter;

public class AgeFileFilterTest {

    public static void main(String[] args) throws IOException {
    String directoryPath = "/home/vikas";
    String fileName = "fileName";
    File directory = new File(directoryPath);


    File[] files = directory.listFiles();
    System.out.println("\nBefore ");
    displayFiles(directory, new AgeFileFilter(new File(directoryPath+FileName) , true));
    System.out.println("\nAfter " );
    displayFiles(directory, new AgeFileFilter(new File(directoryPath+FileName),false));

}

public static void displayFiles(File directory, FileFilter fileFilter) {
    File[] files = directory.listFiles(fileFilter);
    Arrays.sort(files, LastModifiedFileComparator.LASTMODIFIED_COMPARATOR);
    for (File file : files) {
        Date lastMod = new Date(file.lastModified());
        System.out.println("File: " + file.getName() + ", Date: " + lastMod + "");
    }
}

}

Java: Get a list of new files from a directory

5 Answers5