Python: Too many values to unpack (what does the os.walk function return)

Question

Basically my question boils down to "What does os.walk actually return?"

I'm carrying out a simple experiment:

list(os.walk('test_dir'))[0]

yields the following result:

('test_dir', [], ['somefile1', 'somefile2'])

So it does seem that the os.walk function in this case returns a generator of a tuple. I can also write something like this:

for root, dirs, files in os.walk('test_dir'):
    pass

but if I, say, do the following:

for root, dirs, files in (item for item in list(os.walk('test_dir'))[0]):
    pass

I will get the above-mentioned error: "ValueError: too many values to unpack (expected 3)"

I don't understand why.

In the second example you are iterating over the three elements in `list(os.walk('test_dir')[0]` (i.e. the path, directory names, and file names of the first node in the dir tree) one by one, so in the first iteration, the path name will be unpacked to `root`, `dirs`, and `files`, in the second iteration the list of directory names and in the third iteration the list of file names. — Heike, Aug 22 '19 at 15:16
Related: [Do I understand os.walk right?](https://stackoverflow.com/q/10989005/4518341) — wjandrea, Aug 22 '19 at 15:37
@wjandrea Corrected the typo. It is related but it by no means answers the question. — igor, Aug 22 '19 at 15:39
@Heike So basically it tries unpacking `test_dir` to root, dirs, files... — igor, Aug 22 '19 at 15:40
BTW, the more efficient way of getting the first element of a generator is `next(gen)`, instead of `list(gen)[0]`, since it just gets the next element instead of making a list of all the elements then selecting the first one. — wjandrea, Aug 22 '19 at 15:49
Also BTW, `(item for item in iterable)` is redundant. Just use `iterable`. — wjandrea, Aug 22 '19 at 15:54

score 0 · Accepted Answer · answered Aug 22 '19 at 15:45

You already know that list(os.walk(...))[0] == ('test_dir', [], ['somefile1', 'somefile2']), which is a tuple.

This code:

for a, b, c in thing:
    ...

...is the same as this:

for something in thing:
    a, b, c = something
    ...

This part: a, b, c = something is called "iterable unpacking".

Note that the loop will attempt to iterate over each element of thing. In your case, it'll try to iterate over each of the three elements of your tuple, and on the first iteration something == 'test_dir', which is an iterable that contains 8 elements.

However, a, b, c = something expects it to have exactly three elements. So, you get an error saying that there were too many values to unpack (expected 3 but got 8).

And how come it still works for the `for a, b, c in os.walk` case? — igor, Aug 22 '19 at 16:02
@igor, `os.walk` returns a list of tuples, each of which contains three elements, so `something` would be that tuple, which is exactly what `a, b, c = something` expects — ForceBru, Aug 22 '19 at 19:39

Python: Too many values to unpack (what does the os.walk function return)

1 Answers1