37

I'm want a function to return a list with directories with a specified path and a fixed depth and soon realized there a few alternatives. I'm using os.walk quite a lot but the code started to look ugly when counting the depth etc.

What is really the most "neat" implementation?

StefanE
  • 7,578
  • 10
  • 48
  • 75

6 Answers6

62

If the depth is fixed, glob is a good idea:

import glob,os.path
filesDepth3 = glob.glob('*/*/*')
dirsDepth3 = filter(lambda f: os.path.isdir(f), filesDepth3)

Otherwise, it shouldn't be too hard to use os.walk:

import os,string
path = '.'
path = os.path.normpath(path)
res = []
for root,dirs,files in os.walk(path, topdown=True):
    depth = root[len(path) + len(os.path.sep):].count(os.path.sep)
    if depth == 2:
        # We're currently two directories in, so all subdirs have depth 3
        res += [os.path.join(root, d) for d in dirs]
        dirs[:] = [] # Don't recurse any deeper
print(res)
phihag
  • 278,196
  • 72
  • 453
  • 469
  • your first example using glob is brilliant and I would never have gone that track. Thanks a million! – StefanE Aug 23 '11 at 11:32
  • You can directly specify your path pattern in the defintion of `filesDepth3`. The current configuration will take very long time if you are browsing through a large folders database. Getting first all 3rd level folders and subsequently filter by names or content is sub-optimal resources use. – mosegui Dec 05 '18 at 12:55
4

A simple, recursive solution using os.scandir:

def _walk(path, depth):
    """Recursively list files and directories up to a certain depth"""
    depth -= 1
    with os.scandir(path) as p:
        for entry in p:
            yield entry.path
            if entry.is_dir() and depth > 0:
                yield from _walk(entry.path, depth)
Gregor Sturm
  • 2,792
  • 1
  • 25
  • 34
4

This is not exactly neat, but under UNIX-like OS, you could also rely on a system tool like "find", and just execute it as an external program, for example:

from subprocess import call
call(["find", "-maxdepth", "2", "-type", "d"])

You can then redirect the output to some string variable for further handling.

Marek Waligórski
  • 144
  • 3
3

I really like phihag's answer. I adapted it to suit my needs. 

import fnmatch,glob
def fileNamesRetrieve( top, maxDepth, fnMask  ):
    someFiles = []
    for d in range( 1, maxDepth+1 ):
        maxGlob = "/".join( "*" * d )
        topGlob = os.path.join( top, maxGlob )
        allFiles = glob.glob( topGlob )
        someFiles.extend( [ f for f in allFiles if fnmatch.fnmatch( os.path.basename( f ), fnMask ) ] )
    return someFiles

I guess I could also make it a generator with something like this:

def fileNamesRetrieve( top, maxDepth, fnMask  ):
    for d in range( 1, maxDepth+1 ):
        maxGlob = "/".join( "*" * d )
        topGlob = os.path.join( top, maxGlob )
        allFiles = glob.glob( topGlob )
        if fnmatch.fnmatch( os.path.basename( f ), fnMask ):
            yield f

Critique welcome.

John Schmitt
  • 1,148
  • 17
  • 39
1

Here's a simple function to do this

import os
from glob import glob
from pathlib import Path

def find_sub_dirs(path, depth=2):
    path = Path(path)
    assert path.exists(), f'Path: {path} does not exist'
    depth_search = '*/' * depth
    search_pattern = os.path.join(path, depth_search)
    return list(glob(f'{search_pattern}'))
Deep Patel
  • 619
  • 7
  • 8
0

Note that @phihag's great suggestion also works with pathlib.Path.glob:

from pathlib import Path
from typing import List

def get_subitems(folder: Path, level: int) -> List[Path]:
    if level == 0:
        return [item]
    pattern = "/".join(["*"] * level)
    return sorted(item for item in item.glob(pattern))
Chris_128
  • 347
  • 5
  • 10