Pythonic iteration over multiple lists in parallel

Question

I'd like to know an elegant, Pythonic way to iterate over a list of lists (or dict of lists) in parallel in Python 3. The number of lists is not known until runtime, so I believe I cannot simply supply them as arguments to the zip() function.

For example, given the following data structure:

var = [['x1' ,'x2' ,'x3'], ['y1', 'y2', 'y3'], ['z1', 'z2', 'z3'], …]

I would like to be able to access the following values on each iteration:

x1,y1,z1 followed by x2,y2,z2 followed by x3,y3,z3 and so on.

Presumably I could achieve this using list indexes directly, or by using itertools.chain, but I'm looking for a more elegant approach using generators or otherwise.

related: [Matrix Transpose in Python](http://stackoverflow.com/q/4937491/4279) — jfs, Feb 20 '14 at 17:31

Hugh Bothwell · Accepted Answer · 2014-02-20T17:16:09.350

8

zip(*var) will automatically unpack your list of lists.

So, for example:

var = [['x1' ,'x2' ,'x3'], ['y1', 'y2', 'y3'], ['z1', 'z2', 'z3'], ['w1', 'w2', 'w3']]

for ltrs in zip(*var):
    print(", ".join(ltrs))

results in

x1, y1, z1, w1
x2, y2, z2, w2
x3, y3, z3, w3

Edit: per comments below, he wants to use the items from a dictionary,

var = {
    'id_172': ['x1', 'x2', 'x3'],
    'id_182': ['y1', 'y2', 'y3'],
    'id_197': ['z1', 'z2', 'z3']
}

I assume we are using the values with the keys in sorted order:

keys = sorted(var.keys())
for ltrs in zip(*(var[k] for k in keys)):
    print(", ".join(ltrs))

which gives

x1, y1, z1
x2, y2, z2
x3, y3, z3

Warning: do note that this sorts the keys in lexocographic order (ie string alphabetical order), so for example "id_93" comes after "id_101". If your labels need to be sorted in numeric order you will need to use a custom key function, something like

keys = sorted(var.keys(), key=lambda k: int(k[3:]))

edited Feb 20 '14 at 17:16

answered Feb 20 '14 at 14:59

Hugh Bothwell

55,315
8
84
99

Wonderfully, hilariously and embarrassingly elegant. Many thanks. – Bede Constantinides Feb 20 '14 at 15:17
Something I also mentioned in the question was the case of a dict of lists. Do you know how I can access in parallel both the keys and values (which contain lists) of a dict in this manner? Presumably I cannot use `.items()` in conjunction with `zip()`? – Bede Constantinides Feb 20 '14 at 16:27
@bedeabc: can you give a sample dict input and output? – Hugh Bothwell Feb 20 '14 at 16:27
Many thanks @hugh. Something like `{'id_172': ['x1', 'x2', 'x3'], 'id_182': ['y1', 'y2', 'y3'], 'id_197': ['z1', 'z2', 'z3']}` – Bede Constantinides Feb 20 '14 at 16:35
1

@bedeabc: two issues: first, a dict has no ordering; do you want the values in any particular order? second, do you want to do anything with the keys, or can they be ignored? – Hugh Bothwell Feb 20 '14 at 16:37
I solved my issue using lists of lists - I was merely curious as to whether there would be a way to access dict keys. Thanks for your comprehensive and obscenely fast answer :) – Bede Constantinides Feb 20 '14 at 17:50
@bedeabc: if you want the keys, they are in `keys`; I have trouble seeing how you would include them as part of the data otherwise. Give me an example? – Hugh Bothwell Feb 20 '14 at 17:57

score 2 · Answer 2 · answered Feb 20 '14 at 15:00

If you look at the input and output data carefully, you are actually transposing a two dimensional array. That can be done easily with Python's builtin zip function, like this

var = [['x1' ,'x2' ,'x3'], ['y1', 'y2', 'y3'], ['z1', 'z2', 'z3']]
print list(zip(*var))
# [('x1', 'y1', 'z1'), ('x2', 'y2', 'z2'), ('x3', 'y3', 'z3')]

You can iterate over it, like this

for x, y, z in zip(*var):
   print x, y, z

Output

x1 y1 z1
x2 y2 z2
x3 y3 z3

Pythonic iteration over multiple lists in parallel

2 Answers2

Linked