Iterating over .wav files in subdirectories of parent directory

Question

Cheers everybody,

I need help with something in python 3.6 exactly. So i have structure of data like this:

  |main directory
  |    |subdirectory's(plural)
  |    |      |.wav files

I'm currently working from a directory where main directory is placed so I don't need to specify paths before that. So firstly I wanna iterate over my main directory and find all subdirectorys. Then in each of them I wanna find the .wav files, and when done with processing them I wanna go to next subdirectory and so on until all of them are opened, and all .wav files are processed. Exactly what I wanna do with those .wav files is input them in my program, process them so i can convert them to numpy arrays, and then I convert that numpy array into some other object (working with tensorflow to be exact, and wanna convert to TF object). I wrote about the whole process if anybody has any fast advices on doing that too so why not. I tried doing it with for loops like:

for subdirectorys in open(data_path, "r"):
    for files in subdirectorys:
        #doing some processing stuff with the file

The problem is that it always raises error 13, Permission denied showing on that data_path I gave him but when I go to properties there it seems okay and all permissions are fine. I tried some other ways like with os.open or i replaced for loop with:

with open(data_path, "r") as data:

and it always raises permission denied error. os.walk works in some way but it's not what I need, and when i tried to modify it id didn't give errors but it also didnt do anything. Just to say I'm not any pro programmer in python so I may be missing an obvious thing but ehh, I'm here to ask and learn. I also saw a lot of similiar questions but they mainly focus on .txt files and not specificaly in my case so I need to ask it here. Anyway thanks for help in advance.

[`glob.glob`](https://docs.python.org/3/library/glob.html#glob.glob) ? Since you are using Python 3.4+, you might want to use [`pathlib.glob`](https://docs.python.org/3/library/pathlib.html#pathlib.Path.glob) — G_M, Sep 15 '18 at 00:51
@g_m is correct, that would be the preferred way. note that the reason you're getting the permission denied error is because you're not allowed to open a directory for reading, like you can a file (in python, this is allowed by some OSs and programming languages). a quick google shows python doesn't even expose the underlying unix opendir function like some other scripting languages do. — Mark Harviston, Sep 15 '18 at 00:54

Jacob Birkett · Accepted Answer · 2018-09-15T01:39:00.273

Edit: If you want an example for glob (more sane), here it is:

from pathlib import Path

# The pattern "**" means all subdirectories recursively,
# with "*.wav" meaning all files with any name ending in ".wav".
for file in Path(data_path).glob("**/*.wav"):
    if not file.is_file():  # Skip directories
        continue

    with open(file, "w") as f:
        # do stuff

For more info see Path.glob() on the documentation. Glob patterns are a useful thing to know.

Previous answer:

Try using either glob or os.walk(). Here is an example for os.walk().

from os import walk, path

# Recursively walk the directory data_path
for root, _, files in walk(data_path):
    # files is a list of files in the current root, so iterate them
    for file in files:
        # Skip the file if it is not *.wav
        if not file.endswith(".wav"):
            continue

        # os.path.join() will create the path for the file
        file = path.join(root, files)
        # Do what you need with the file
        # You can also use block context to open the files like this
        with open(file, "w") as f:  # "w" means permission to write. If reading, use "r"
             # Do stuff

Note that you may be confused about what open() does. It opens a file for reading, writing, and appending. Directories are not files, and therefore cannot be opened.

I suggest that you Google for documentation and do more reading about the functions used. The documentation will help more than I can.

Another good answer explaining in more detail can be seen here.

I was just reading through the `pathlib` docs and noticed [`Path.rglob`](https://docs.python.org/3/library/pathlib.html#pathlib.Path.rglob) too. It only saves a couple characters but I just wanted to leave it here for the future. — G_M, Sep 15 '18 at 01:43
Thank you sir for help. I haven't tried other people's code because this one does what I need. Although maybe I will need to modify other examples later but this one currently fits the best for my use. I think the reason I was getting permission denied error was not that my directories or files weren't accessible than my wrong code, like I was trying to open a directory as you said isn't possible and is used only for opening files. — deva56, Sep 16 '18 at 14:59
@deva56 Obviously I'm biased, but this solution is far more elegant than others (some other answers were even deleted). This is the "correct" way of doing such a thing. The one other answer that is still up has some flaws. It's not very readable and doesn't seem to follow "The Zen of Python", it also isn't performant. — Jacob Birkett, Sep 16 '18 at 18:46

score 1 · Answer 2 · answered Sep 15 '18 at 01:14

1

import glob
import os

main = '/main_wavs'
wavs = [w for w in glob.glob(os.path.join(main, '*/*.wav')) if os.path.isfile(w)]

In terms of permissions on a path A/B/C... A, B and C must all be accessible. For files that means read permission. For directories, it means read and execute permissions (listing contents).

answered Sep 15 '18 at 01:14

jjak_82

46
3

1

There is no reason to use list comprehension for this. Making a list of the files only means that OP has to iterate over a list twice, once to create it, and again to use the files. – Jacob Birkett Sep 15 '18 at 01:17

Iterating over .wav files in subdirectories of parent directory

2 Answers2