How to flatten zipped object containing nested arrays to a list?

Question

I have a zipped object like this:

z = zip(a, b)
lst = list(z)

print(lst)

Output:

[(0, array([[72, 65],
           [70, 71]], dtype=uint8)), 
(1, array([[ 71,  99],
           [190, 163]], dtype=uint8)), 
(2, array([[52, 59],
           [69, 72]], dtype=uint8)), etc...

I would like to flatten this list to the following:

[0, 72, 65, 70, 71, 1, 71, 99, 190, 163, 2, 52, 59 etc..]

I've tried doing this with

y = sum(w, ())
# or 
y = list(itertools.chain(*lst))

But the arrays are still there when I print.

What am I doing wrong?

Pay attention to the context in which you saw that advice. You can't just blindly apply it anywhere, you need to understand why it works in those cases. — Alex Hall, May 30 '16 at 22:47

score 5 · Accepted Answer · edited May 23 '17 at 10:28

Use one of

numpy.ravel returns a contiguous flattened array.
numpy.ndarray.flat returns a 1-D iterator over the array. (this is a numpy.flatiter instance)
numpy.ndarray.flatten that returns a copy of the array collapsed into one dimension.

Here is a MWE.

import numpy as np

lists = [(0, np.array([ [72,  65],
                        [70,  71]], dtype=np.uint8)), 
        (1, np.array([  [71,  99],
                        [190, 163]], dtype=np.uint8))]

l = list()
for idx, array in lists:
    l.append(idx)
    l.extend(np.ravel(array))   # returns a contiguous flattened array
    #l.extend(array.flat)       # return a 1-D iterator over the array.
    #l.extend(array.flatten())  # return a copy of the array collapsed into one dimension 

print(l)
# Output
[0, 72, 65, 70, 71, 1, 71, 99, 190, 163]

In speaking of the differences between ravel and flatten, excerpt from What is the difference between flatten and ravel functions in numpy?,

The difference is that flatten always returns a copy and ravel returns a view of the original array whenever possible. This isn't visible in the printed output, but if you modify the array returned by ravel, it may modify the entries in the original array. If you modify the entries in an array returned from flatten this will never happen. ravel will often be faster since no memory is copied, but you have to be more careful about modifying the array it returns.

You don't need tolist, you don't actually need flatten either, you can `array.flat`, extend takes an iterable — Padraic Cunningham, May 30 '16 at 22:53
@PadraicCunningham, thx for pointing out this. I just edited my answer. — SparkAndShine, May 30 '16 at 22:56
You can also use `.ravel` if you are going to extend, ravel does not always need to make a copy — Padraic Cunningham, May 30 '16 at 22:57

Padraic Cunningham · Answer 2 · 2016-05-30T23:04:38.110

3

You data is nested so you would need chain to recursively flatten which chain does not do, what you can do is flatten the numpy arrays and add:

from itertools import chain

print list(chain.from_iterable(([a] + arr.flatten().tolist()) for a, arr in lst))

Or use .flat and an inner chain:

print list(chain.from_iterable(chain([a],  arr.flat) for a, arr in lst))

edited May 30 '16 at 23:04

answered May 30 '16 at 22:45

Padraic Cunningham

176,452
29
245
321

score 0 · Answer 3 · answered May 30 '16 at 22:45

0

from itertools import chain
list(chain.from_iterable(chain([x], chain.from_iterable(y)) for x, y in z))

I'm using just chain for consistency but there are plenty of ways to flatten things, as @Padriac has shown.

answered May 30 '16 at 22:45

Alex Hall

34,833
5
57
89

How to flatten zipped object containing nested arrays to a list?

3 Answers3