How does the iteration of os.walk work in Python 3?

Question

According to the Python 3 docs os.walk returns a 3-tuple. However, this does not work:

root, dirs, files = os.walk('path')

Neither does this:

(root, dirs, files) = os.walk('path')

It always yields:

ValueError: not enough values to unpack (expected 3, got 1)

All the examples for os.walk I found embed os.walk in a for loop:

for root, dirs, files in os.walk('path'):

Why? What exactly will be iterated here? root is a string, dirs and files are lists. However, most examples iterate again over dirs and files:

for root, dirs, files in os.walk('path'):
    for name in files:
        print(name)
    for name in dirs:
        print(name)

The inner for loops make sense to me, but I don't get what the outer for loop is for.

And why does the assignment to the 3-tuple work when os.walk is embedded in a for loop, but not without one?

I seems to yield root, dirs and files. So if you are in A, it will yield /A, dirsinsideA and filesinsideA. Next iteration will yield the first subdirectory. It is my guess. — JorgeeFG, Mar 05 '19 at 18:18
Just because you can do `for x,y,z in some_iterable` does **not mean** you can do `x,y,z = some_iterable`. In fact, that is almost never the case, it only will ever work if `some_iterable` contains exactly three objects — juanpa.arrivillaga, Mar 05 '19 at 18:20
Possible duplicate of [Do I understand os.walk right?](https://stackoverflow.com/questions/10989005/do-i-understand-os-walk-right) — wjandrea, Mar 26 '19 at 19:31

wjandrea · Accepted Answer · 2019-03-07T17:52:32.327

5

os.walk doesn't return a 3-tuple, it yields multiple 3-tuples. From the docs:

For each directory in the tree rooted at directory top (including top itself), it yields a 3-tuple (dirpath, dirnames, filenames).

For how yielding works, see What does the "yield" keyword do? (You can ignore the example code given by OP.) Technically os.walk returns a generator that yields the 3-tuples.

The outer for loop iterates over the 3-tuples. You're probably confused by the unpacking that happens in the same step. So to be clear,

for dirpath, dirnames, filenames in os.walk(top):

is effectively the same as

for branch in os.walk(top):
    dirpath, dirnames, filenames = branch

edited Mar 07 '19 at 17:52

answered Mar 05 '19 at 18:16

wjandrea

28,235
9
60
81

Could you please expand on that? What's the difference between returning and yielding? – user1785730 Mar 05 '19 at 18:17
2

Yielding is for iterators. Instead of returning a full length array, it will return only the needed for the iteration, thus saving memory space. – JorgeeFG Mar 05 '19 at 18:20
1

@user1785730 See [What does the “yield” keyword do?](https://stackoverflow.com/q/231767/4518341) (You can ignore the example code OP gives.) – wjandrea Mar 05 '19 at 18:20
Thanks, that was an interesting read and answers most of my questions. The one remaining is: What are the values the outer for loop iterates over? – user1785730 Mar 05 '19 at 18:29
@user1785730 I added that to my answer – wjandrea Mar 05 '19 at 18:40

score 2 · Answer 2 · answered Mar 05 '19 at 18:20

2

os.walk returns an generator not a tuple! If you want to look up values from your generator output convert it to a list:

test = list(os.walk("."))

answered Mar 05 '19 at 18:20

mtt2p

1,818
1
15
22

How does the iteration of os.walk work in Python 3?

2 Answers2