3

I am trying to walk file tree to count files. But even though I handle exceptions within my CountFiles class the call to walkFileTree() has failed prematurely i the FileTreeWalker class preventing it counting the remaining files.

How can I avoid this ?

countFiles = new CountFiles(BaseFolderGuesser.FILE_SUFFIX_SEARCH_PATTERN);
Files.walkFileTree(path, countFiles);
totalCount+=countFiles.getFileCount();

public static class CountFiles
            extends SimpleFileVisitor<Path>
    {
        private int fileCount = 0;
        private Pattern pattern;

        public CountFiles(String regex)
        {
            pattern = Pattern.compile(regex);
        }

        /**
         * SONGKONG-294  Ignore the /proc virtual fs on linux
         *
         * @param dir
         * @param attrs
         * @return
         * @throws IOException
         */
         /*
         * Ignore some dirs
         * @param dir
         * @param attrs
         * @return
         * @throws IOException
         */
        public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
                throws IOException
        {

            if (dir.toString().equals("/proc")) {
                MainWindow.logger.log(Level.SEVERE,"Ignoring /proc");
                return FileVisitResult.SKIP_SUBTREE;
            }
            else if (RecycleBinFolderNames.isMatch(dir.toFile().getName()))
            {
                MainWindow.logger.log(Level.SEVERE,"Ignoring "+dir.toString());
                return FileVisitResult.SKIP_SUBTREE;
            }
            return super.preVisitDirectory(dir, attrs);
        }

        /**
         * Find Music file
         *
         * @param file
         * @param attr
         * @return
         */
        @Override
        public FileVisitResult visitFile(Path file,
                                         BasicFileAttributes attr)
        {
            Path name = file.getFileName();
            if (name != null && pattern.matcher(name.toString().toLowerCase(Locale.UK)).matches())
            {
                fileCount++;
            }
            return FileVisitResult.CONTINUE;
        }

        /**
         * http://stackoverflow.com/questions/14436032/why-is-java-7-files-walkfiletree-throwing-exception-on-encountering-a-tar-file-o/14446993#14446993
         * SONGKONG-294:Ignore exceptions if file is not readable
         *
         * @param file
         * @param exc
         * @return
         * @throws IOException
         */
        @Override
        public FileVisitResult visitFileFailed(Path file, IOException exc) throws IOException {

            if (file.toString().endsWith(".tar")) {
                //We dont log to reports as this is a bug in Java that we are handling not a problem in SongKong
                MainWindow.logger.log(Level.SEVERE, exc.getMessage());
                return FileVisitResult.CONTINUE;
            }

            try
            {
                FileVisitResult result = super.visitFileFailed(file, exc);
                return result;
            }
            catch(AccessDeniedException ade)
            {
                MainWindow.logger.warning("Unable to count files in:"+file);
                return FileVisitResult.CONTINUE;
            }
        }

        /**
         * SONGKONG-294:Ignore exception if folder is not readable
         *
         * @param dir
         * @param exc
         * @return
         * @throws IOException
         */
        @Override
        public FileVisitResult postVisitDirectory(Path dir, IOException exc)
                throws IOException
        {
            try
            {
                FileVisitResult result = super.postVisitDirectory(dir, exc);
                return result;
            }
            catch(AccessDeniedException ade)
            {
                MainWindow.logger.warning("Unable to count files in dir:"+dir);
                return FileVisitResult.CONTINUE;
            }
        }


        public int getFileCount()
        {
            return fileCount;
        }
    }

Giving Error

  13/09/2017 11.25.11:EDT:com.jthink.songkong.fileloader.CountFilesinFolder:handleException:SEVERE: Unable to count files:/Volumes/PlexMedia/Music/White Stripes, The - De Stijl/1BN0PB~Y
    java.nio.file.NoSuchFileException: /Volumes/PlexMedia/Music/White Stripes, The - De Stijl/1BN0PB~Y
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
    at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
    at java.nio.file.Files.readAttributes(Files.java:1737)
    at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
    at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
    at java.nio.file.FileTreeWalker.next(FileTreeWalker.java:372)
    at java.nio.file.Files.walkFileTree(Files.java:2706)
    at java.nio.file.Files.walkFileTree(Files.java:2742)
    at com.jthink.songkong.fileloader.CountFilesinFolder.call(CountFilesinFolder.java:174)
    at com.jthink.songkong.fileloader.CountFilesinFolder.call(CountFilesinFolder.java:26)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:748)
Paul Taylor
  • 13,411
  • 42
  • 184
  • 351
  • @pvg Files.walkFileTree(path, countFiles); – Paul Taylor Sep 13 '17 at 17:49
  • @pvg I don't know how yo came to that wrong conclusion, the stack trace clearly shows the error occurs in Files.walkFileTree(), that is a System class, the only way I can affect that is with the FileVisitor instance which is the CountFiles class I have pasted, but its looks like the problem is occurring in the code System code anyway. The problem does not occur for me it is specific to a user due to a particular file but I hoped somebody would have an idea on how to work round this rather than just spouting the unhelpful ' a Minimal, Complete, and Verifiable example line'. – Paul Taylor Sep 13 '17 at 20:00
  • 1
    Let's try to avoid the personal attacks, please, and focus only on the technical issues. – Cody Gray - on strike Sep 13 '17 at 22:03
  • Its look to me like a permissions issue, but I cant see how to get round it. Concurrency could be an issue as I start processing files immediatley rather than waiting for them all to be counted first but I dont think it is concurrency. I cannot reproduce the error , if I could I i would not have needed to post on stackoverflow. – Paul Taylor Sep 14 '17 at 06:32
  • Is this file really exists? (/Volumes/PlexMedia/Music/White Stripes, The - De Stijl/1BN0PB~Y) It seems like it is temporary file that was deleted while program was running. – A.Alexander Sep 16 '17 at 13:32
  • @A.Alexander good point, i dont know, but even if that is the case how can I prevent FileWalker falling over, and this issue could always happen – Paul Taylor Sep 17 '17 at 17:39
  • @pvg if the answer below is correct it seems that me posting visitor instance is not a distraction after all ! – Paul Taylor Sep 18 '17 at 20:24

1 Answers1

3

Your class extends SimpleFileVisitor and in the overridden visitFileFailed method you wrote:

        try
        {
            FileVisitResult result = super.visitFileFailed(file, exc);
            return result;
        }
        catch(AccessDeniedException ade)
        {
            MainWindow.logger.warning("Unable to count files in:"+file);
            return FileVisitResult.CONTINUE;
        }

You call your super class' visitFileFailed which is:

/**
 * Invoked for a file that could not be visited.
 *
 * <p> Unless overridden, this method re-throws the I/O exception that prevented
 * the file from being visited.
 */
@Override
public FileVisitResult visitFileFailed(T file, IOException exc)
    throws IOException
{
    Objects.requireNonNull(file);
    throw exc;
}

Which rethrow the same exception which was handled before. Thatswhy you got the exception. After that you only catch the AccessDeniedException but as I can see you got a NoSuchFileException.

Maybe you can consider catching a broader range of exception or not invoking super.visitFileFailed at all since it does nothing.

Zsolt V
  • 517
  • 3
  • 8
  • Thankyou sounds like you have found the solution I will try it. – Paul Taylor Sep 18 '17 at 20:23
  • COmapring with https://stackoverflow.com/questions/14436032/why-is-java-7-files-walkfiletree-throwing-exception-on-encountering-a-tar-file-o/14446993#14446993 the stack trace is different, in that question the preceding call was walk(), in this one it is visit() so I wonder if the error is actually occurring earlier on and I need to change previsit() method – Paul Taylor Sep 20 '17 at 06:51
  • I think you don't have to change previsit() method just in the case if you can find a pattern how to avoid such directories. Either way, I would definitely modify the visitFileFailed() method to make it more error-prone. – Zsolt V Sep 21 '17 at 19:03