java.nio.file.Files.delete(Path path) - occasional failure to recursively delete directory using SimpleFileVisitor

Question

Trying to troubleshoot an occasional java.nio.file.DirectoryNotEmptyException in a recursive delete method taken from Delete directories recursively in Java

Code (credit to @TrevorRobinson) :

static void removeRecursive(Path path) throws IOException {
    Files.walkFileTree(path, new SimpleFileVisitor<Path>() {

        final Logger logger = LoggerFactory.getLogger(this.getClass());
        @Override
        public FileVisitResult visitFile(Path file,
                BasicFileAttributes attrs) throws IOException {
            logger.warn("Deleting " + file.getFileName());
            Files.delete(file);
            logger.warn("DELETED " + file.getFileName());
            return FileVisitResult.CONTINUE;
        }

        @Override
        public FileVisitResult visitFileFailed(Path file, IOException exc) {
            // try to delete the file anyway, even if its attributes could
            // not be read, since delete-only access is theoretically possible
            // I NEVER SEE THIS
            logger.warn("Delete file " + file + " failed", exc);
            try {
                Files.delete(file);
            } catch (IOException e) {
                logger.warn(
                    "Delete file " + file + " failed again", exc);
            }
            return FileVisitResult.CONTINUE;
        }

        @Override
        public FileVisitResult postVisitDirectory(Path dir, IOException exc)
                throws IOException {
            if (exc == null) {
                Files.delete(dir);
                return FileVisitResult.CONTINUE;
            }
            // directory iteration failed; propagate exception
            throw exc;
        }
    });
}

Call:

try {
    removeRecursive(Paths.get(unzipDirPath));
} catch (IOException e) {
    String msg = "Failed to delete folder " + unzipDirPath;
    if (e instanceof java.nio.file.DirectoryNotEmptyException) {
        msg += ". Still contains : ";
        final File[] listFiles = Paths.get(unzipDirPath).toFile().listFiles();
        if (listFiles != null) for (File file : listFiles) {
            msg += file.getAbsolutePath() + "\n";
        }
    }
    log.error(msg, e);
}

Prints (once in 20/40 iterations):

22:03:34.190 [http-bio-8080-exec-47] WARN  g.u.d.m.server.servlets.Controller$1 - Deleting batt
22:03:34.192 [http-bio-8080-exec-47] WARN  g.u.d.m.server.servlets.Controller$1 - DELETED batt
22:03:34.192 [http-bio-8080-exec-47] WARN  g.u.d.m.server.servlets.Controller$1 - Deleting wifi
22:03:34.193 [http-bio-8080-exec-47] WARN  g.u.d.m.server.servlets.Controller$1 - DELETED wifi
22:03:34.196 [http-bio-8080-exec-47] ERROR g.u.d.m.s.s.DataCollectionServlet - Failed to delete folder C:\yada\. Still contains : C:\yada\dir\wifi

java.nio.file.DirectoryNotEmptyException: C:\yada\dir
    at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:265) ~[na:1.7.0_45]
    at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103) ~[na:1.7.0_45]
    at java.nio.file.Files.delete(Files.java:1077) ~[na:1.7.0_45]
    at gr.uoa.di.monitoring.server.servlets.Controller$1.postVisitDirectory(Controller.java:128) ~[Controller$1.class:na]
    at gr.uoa.di.monitoring.server.servlets.Controller$1.postVisitDirectory(Controller.java:1) ~[Controller$1.class:na]
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:224) ~[na:1.7.0_45]
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:199) ~[na:1.7.0_45]
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:69) ~[na:1.7.0_45]
    at java.nio.file.Files.walkFileTree(Files.java:2600) ~[na:1.7.0_45]
    at java.nio.file.Files.walkFileTree(Files.java:2633) ~[na:1.7.0_45]
    at gr.uoa.di.monitoring.server.servlets.Controller.removeRecursive(Controller.java:96) ~[Controller.class:na]
    at gr.uoa.di.monitoring.server.servlets.DataCollectionServlet.doPost(DataCollectionServlet.java:153) ~[DataCollectionServlet.class:na]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:641) [servlet-api.jar:na]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:722) [servlet-api.jar:na]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) [catalina.jar:7.0.32]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) [catalina.jar:7.0.32]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) [catalina.jar:7.0.32]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) [catalina.jar:7.0.32]
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) [catalina.jar:7.0.32]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168) [catalina.jar:7.0.32]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) [catalina.jar:7.0.32]
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929) [catalina.jar:7.0.32]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) [catalina.jar:7.0.32]
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) [catalina.jar:7.0.32]
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1002) [tomcat-coyote.jar:7.0.32]
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:585) [tomcat-coyote.jar:7.0.32]
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310) [tomcat-coyote.jar:7.0.32]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_45]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45]
    at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]

Notice that wifi is reported as deleted - what's even more weird is that sometimes I get :

Failed to delete folder C:\yada. Still contains : C:\yada\dir

java.nio.file.DirectoryNotEmptyException: C:\yada\dir

I tend towards the conclusion that occasionally the deletion takes too long - in other words the problem is that java.nio.file.Files.delete(Path path) does not block (so C:\yada\dir still contains files when its time comes, which sometimes are deleted by the time I stat it). So how am I to workaround this ?

Also : is java.nio.file.Files.delete(Path path) required to throw ? The docs state :

On some operating systems it may not be possible to remove a file when it is open and in use by this Java virtual machine or other programs.

Does not seem to require an exception to be thrown in this case. Is java.nio.file.Files.delete(Path path) required to throw ?

Answer 1

I was having the same problem and it turned out the problem was caused by an unclosed directory file stream somewhere else in the code for the same directory I was deleting stuff from. The stream object returned by:

Files.list(Path)

must be closed, so be sure to use the try-with-resources construct in your code if using that method.

So I don't think it's that the deletion takes too long, I tried a wait before re-attempting directory deletion without any luck. It's most likely that your own program has that resource locked. The result is that the delete call on it doesn't complete although it returns successfully (it looks like Windows will eventually delete the file once your own program frees it) but then of course the containing directory cannot be deleted as that is indeed not yet empty.

Answer 2

I know this is a very old thread - but I had the same problem and it took me quite some time to get it fixed. I think this misbehaviour is caused by a timing issue (looks like it happens on Windows only), so I put a pause into the postVisitDirectory method. That worked and this is what I finally came up with:

A method that does the delete without throwing the DirectoryNotEmptyException:

private boolean isDeleted(Path dir) throws IOException {
    boolean deleted = false;
    try {
        Files.delete(dir);
        deleted = true;
    } catch (DirectoryNotEmptyException e) {
    // happens sometimes if Windows is too slow to remove children of a directory
    deleted = false;
    }
    return deleted;
}

and its use in a loop:

public FileVisitResult postVisitDirectory(Path dir, IOException e) throws IOException {
    if (e == null) {
        int maxTries = 5;
        int count = 0;
        boolean deleted = false;
        do {
            if ((deleted = this.isDeleted(dir))) {
                break;
            } else {
                // wait a bit and try again
                count++;
                try {
                    Thread.sleep(1);
                } catch (InterruptedException e1) {
                    Thread.currentThread().interrupt();
                    break;
                }
            }
        } while (count < maxTries);
        // gone?
        if (!deleted) {
            throw new DirectoryNotEmptyException(dir.toString());
        }
        // go ahead
        return FileVisitResult.CONTINUE;
    }
    throw e;
}

Andy

Answer 3

I would have commented on the answer by @user3485962, but don't have enough points for that!

Here is an example using try with resources that closes the stream after creating a list:

    /**  
     * @param dir The directory to list.
     * @return A list of files and directories in dir.
     * @throws IOException If encountered.
     */
    public static List<Path> getList(Path dir) throws IOException {
        try (Stream<Path> s = Files.list(dir)) {
            return s.collect(Collectors.toList());
        }
    }

Answer 4

You could temporarily store the name of the file you're currently iterating in a variable and perform a try-catch on DirectoryNotEmptyException. When this exception occurs, catch it and throw your own exception that specifies the file.

try {
 Files.delete(file);
} catch (DirectoryNotEmptyException  e) {
 throw new MySpecificException(file.getFileName());
}

class MySpecificException extends Exception {
 public MySpecificException() { }

 public MySpecificException(string filename) {
  super(filename);
 }  
}

Getting the filename can be done with e.getMessage();

I assume Files.delete() continues deleting the files in a directory when it encounters a file that can't be deleted. If this is the case, my method still works: just return a list of all the files in the directory instead of the filename of the directory. It should only contain files that can't be deleted and you still have your solution.

Answer 5

An expansion on my comment above.

When Java has a problem doing something it throws an Exception. Exceptions, like all other types, can be inherited from. The API will specify which checked exceptions each method throws. The methods in java.io and java.nio will usually throw IOException or one of it's children. If you want to create a method that will tell you why the file operation, in this case deletion, failed you can do something like this:

@Override
public FileVisitResult visitFileFailed(Path file, IOException exc) throws IOException {
       //You don't need to try to delete the file again since this method was called
       //because the deletion failed. Instead...
       if (exc instanceof NoSuchFileException) {
            System.out.println("Could not find the file: " + file);
       } else if (exc instanceof DirectoryNotEmptyException) {
            System.out.println("The directory [+ " file + "] was not empty.");
       } else {
            System.out.println("Could not delete file [" + file
                + "]. There was a problem communicating with the file system");
       }

       return FileVisitResult.CONTINUE;
}

You can change the actual program response as you need to but that should give you the general idea.