How to use Files.walk()... to get a graph of files based on conditions?

Question

I have the following directory structures:

/path/to/stuff/org/foo/bar/
/path/to/stuff/org/foo/bar/1.2.3/
/path/to/stuff/org/foo/bar/1.2.3/myfile.ext
/path/to/stuff/org/foo/bar/1.2.4/
/path/to/stuff/org/foo/bar/1.2.4/myfile.ext
/path/to/stuff/org/foo/bar/blah/
/path/to/stuff/org/foo/bar/blah/2.1/
/path/to/stuff/org/foo/bar/blah/2.1/myfile.ext
/path/to/stuff/org/foo/bar/blah/2.2/
/path/to/stuff/org/foo/bar/blah/2.2/myfile.ext

I would like to get the following output:

/path/to/stuff/org/foo/bar/
/path/to/stuff/org/foo/bar/blah/

I have the following code (below), which is inefficient, as it prints out:

/path/to/stuff/org/foo/bar/
/path/to/stuff/org/foo/bar/
/path/to/stuff/org/foo/bar/blah/
/path/to/stuff/org/foo/bar/blah/

Here is the Java code:

public class LocatorTest
{

    @Test
    public void testLocateDirectories()
            throws IOException
    {
        long startTime = System.currentTimeMillis();

        Files.walk(Paths.get("/path/to/stuff/"))
             .filter(Files::isDirectory)
             .forEach(Foo::printIfArtifactVersionDirectory);

        long endTime = System.currentTimeMillis();

        System.out.println("Executed in " + (endTime - startTime) + " ms.");
    }

    static class Foo
    {

        static void printIfArtifactVersionDirectory(Path path)
        {
            File f = path.toAbsolutePath().toFile();
            List<String> filePaths = Arrays.asList(f.list(new MyExtFilenameFilter()));

            if (!filePaths.isEmpty())
            {
                System.out.println(path.getParent());
            }
        }

    }

}

The filter:

public class MyExtFilenameFilter
        implements FilenameFilter
{

    @Override
    public boolean accept(File dir, String name)
    {
        return name.endsWith(".ext");
    }

}

`.allMatch()` doesn't compile. – Seelenvirtuose Apr 11 '15 at 06:01 — Seelenvirtuose, Apr 11 '15 at 06:01
@Seelenvirtuose: Sorry about that. Fixed it. – carlspring Apr 11 '15 at 06:19 — carlspring, Apr 11 '15 at 06:19

score 41 · Accepted Answer · edited Mar 24 '17 at 11:33

41

Files.walk(Paths.get("/path/to/stuff/"))
     .filter(p -> p.toString().endsWith(".ext"))
     .map(p -> p.getParent().getParent())
     .distinct()
     .forEach(System.out::println);

This filters all files that have the extension and gets the parent path of their directory. distinct ensures that every path is used only once.

edited Mar 24 '17 at 11:33

carlspring

31,231
29
115
197

answered Apr 11 '15 at 06:48

a better oliver

26,330
2
58
66

I'd recommend to accept this answer since it's quite elegant and nicely uses Java 8 features. – S. Pauk Apr 11 '15 at 16:23
1

Why not `.filter(p -> p.endsWith(".ext"))`? – user2336315 Apr 13 '15 at 19:48
7

@user2336315 Because it doesn't consider the file part, only the directory part of a path. – a better oliver Apr 13 '15 at 21:45
1

Don't forget to close the stream, with a `try-with-resources` for example. – Theo Jan 27 '20 at 09:33

Seelenvirtuose · Answer 2 · 2015-04-11T06:46:06.073

You are invoking the method printIfArtifactVersionDirectory for all visited directories. I did a little change to make it obvious:

static void printIfArtifactVersionDirectory(Path path) {
    System.out.println("--- " + path);
    ...
}

With that additional output you will get:

--- C:\Projects\stuff
--- C:\Projects\stuff\org
--- C:\Projects\stuff\org\foo
--- C:\Projects\stuff\org\foo\bar
--- C:\Projects\stuff\org\foo\bar\1.2.3
C:\Projects\stuff\org\foo\bar
--- C:\Projects\stuff\org\foo\bar\1.2.4
C:\Projects\stuff\org\foo\bar
--- C:\Projects\stuff\org\foo\bar\blah
--- C:\Projects\stuff\org\foo\bar\blah\2.1
C:\Projects\stuff\org\foo\bar\blah
--- C:\Projects\stuff\org\foo\bar\blah\2.2
C:\Projects\stuff\org\foo\bar\blah

So you get the output as often as you have artifact version directories. If you want to remember that you already did the output for one directory, you must make store this information somewhere. One quick implementation could be:

static class Foo {
    private static final Set<Path> visited = new HashSet<>();

    static void printIfArtifactVersionDirectory(Path path) {
        ...
        Path parent = path.getParent();
        if (!filePaths.isEmpty() && !visited.contains(parent)) {
            visited.add(parent);
            System.out.println(parent);
        }
    }
}

With this you get the expected output:

C:\Projects\stuff\org\foo\bar
C:\Projects\stuff\org\foo\bar\blah

A better solution would be to use the set for storing the visited parents and only print them after visiting them all:

static class PathStore {
    private final Set<Path> store = new HashSet<>();

    void visit(Path path) {
        File f = path.toAbsolutePath().toFile();
        List<String> filePaths = Arrays.asList(f.list(new MyExtFilenameFilter()));
        if (!filePaths.isEmpty()) {
            store.add(path.getParent());
        }
    }

    void print() {
        store.forEach(System.out::println);
    }
}

Usage:

PathStore pathStore = new PathStore();
Files.walk(Paths.get("/path/to/stuff/"))
        .filter(Files::isDirectory)
        .forEach(pathStore::visit);
pathStore.print();

Hi, thanks for this answer! Your answer is more than excellent and I will be using it this way in one of the cases (where there will be follow-up actions). I will however choose the other one, because it will be used like this most of the time, as it much simpler and more elegant. Thank you very much though! — carlspring, Apr 12 '15 at 19:25

score 1 · Answer 3 · answered Dec 17 '19 at 14:11

Adding to @a-better-oliver's answer, here is what you can do if your action declares IOExceptions.

You can treat the filtered stream as an Iterable, and then do your action in a regular for-each loop. This way, you don't have to handle exceptions inside a lambda.

try (Stream<Path> pathStream = Files
        .walk(Paths.get("/path/to/stuff/"))
        .filter(p -> p.toString().endsWith(".ext"))
        .map(p -> p.getParent().getParent())
        .distinct()) {

    for (Path file : (Iterable<Path>) pathStream::iterator) {
        // something that throws IOException
        Files.copy(file, System.out);
    }
}

Found that trick here: https://stackoverflow.com/a/32668807/1207791

How to use Files.walk()... to get a graph of files based on conditions?

3 Answers3

Linked