Concatenation of inner lists or ints

Question

I feel like I'm missing something obvious, but there it is... I would like to go from:

lst = [[0, 1, 3, 7, 8, 11, 12], [8, 0, 1, 2, 3, 14], 2]

to:

output = [0, 1, 3, 7, 8, 11, 12, 8, 0, 1, 2, 3, 14, 2]

I can do this with a for loop such as:

output = []
for l in lst:
    if hasattr(l, '__iter__'):
        output.extend(l)
    else:
        output.append(l)

Maybe the for-loop is fine, but it feels like there should be a more elegant way to do this... Trying to do this with numpy seems even more convoluted because ragged arrays aren't easily handled... so you can't (for example):

output = np.asanyarray(lst).flatten().tolist()

Thanks in advance.

Update:

Here's my comparison between the two methods provided by @T.J and @Ashwini - thanks to both!

In [5]: %paste
from itertools import chain
from collections import Iterable
lis = [[0, 1, 3, 7, 8, 11, 12], [8, 0, 1, 2, 3, 14], 2]
def solve(lis):
    for x in lis:
        if isinstance(x, Iterable) and not isinstance(x, basestring):
            yield x
        else:
            yield [x]

%timeit list(chain.from_iterable(solve(lis)))

%timeit [a for x in lis for a in (x if isinstance(x, Iterable) and not isinstance(x,basestring) else [x])]
## -- End pasted text --
100000 loops, best of 3: 10.1 us per loop
100000 loops, best of 3: 8.12 us per loop

Update2:

...
lis = lis *10**5
%timeit list(chain.from_iterable(solve(lis)))

%timeit [a for x in lis for a in (x if isinstance(x, Iterable) and not isinstance(x,basestring) else [x])]
## -- End pasted text --
1 loops, best of 3: 699 ms per loop
1 loops, best of 3: 698 ms per loop

...or for irregular lists, try [this](http://stackoverflow.com/questions/2158395/flatten-an-irregular-list-of-lists-in-python). — Aya, Jun 21 '13 at 19:58
Yup, sure is - apologies for the dup, didn't think to search 'flatten' :/ — blazetopher, Jun 21 '13 at 20:09
Probably because (on Python 2.x) you can just do `from compiler.ast import flatten; flatten([[0, 1, 3, 7, 8, 11, 12], [8, 0, 1, 2, 3, 14], 2])` -> `[0, 1, 3, 7, 8, 11, 12, 8, 0, 1, 2, 3, 14, 2]` — Aya, Jun 21 '13 at 20:13
Change the list size before timing the solutions: `lis = lis *10**5`, `itertools.chain` is faster than normal list comprehension for bigger size list, for small lists timings doesn't matter. — Ashwini Chaudhary, Jun 21 '13 at 20:36
Very good point - looks like `lis=lis *10**5` is pretty much the tipping point between the two methodologies. — blazetopher, Jun 21 '13 at 20:42
@blazetopher well that's weird, itertools.chain is faster for moderate number of items too. And in-fact it is preferred over normal list comprehension for flattening a list. see : http://stackoverflow.com/a/408281/846892 — Ashwini Chaudhary, Jun 22 '13 at 21:39

Ashwini Chaudhary · Answer 1 · 2013-06-21T20:37:25.717

2

You can use itertools.chain like this:

>>> from itertools import chain
>>> from collections import Iterable
>>> lis = [[0, 1, 3, 7, 8, 11, 12], [8, 0, 1, 2, 3, 14], 2]
def solve(lis):
    for x in lis:
        if isinstance(x, Iterable) and not isinstance(x, basestring):
            yield x
        else:
            yield [x]
...             

>>> list(chain.from_iterable(solve(lis)))
[0, 1, 3, 7, 8, 11, 12, 8, 0, 1, 2, 3, 14, 2]

Works fine for strings too:

>>> lis = [[0, 1, 3, 7, 8, 11, 12], [8, 0, 1, 2, 3, 14], "234"]
>>> list(chain.from_iterable(solve(lis)))
[0, 1, 3, 7, 8, 11, 12, 8, 0, 1, 2, 3, 14, '234']

Timing comparisons:

>>> lis = lis *(10**4)
#modified version of FJ's answer that works for strings as well
>>> %timeit [a for x in lis for a in (x if isinstance(x, Iterable) and not isinstance(x,basestring) else [x])]
10 loops, best of 3: 110 ms per loop

>>> %timeit list(chain.from_iterable(solve(lis)))
1 loops, best of 3: 98.3 ms per loop

edited Jun 21 '13 at 20:37

answered Jun 21 '13 at 19:56

Ashwini Chaudhary

244,495
58
464
504

Thanks, this is a nice solution. Any idea if this is more/less performant than @F.J's answer? – blazetopher Jun 21 '13 at 20:05
@blazetopher FJ's solution is going to be faster as he's using a list comprehension , I used a function to make the solution more readable. And my solution also handles the case if strings are present in the data. – Ashwini Chaudhary Jun 21 '13 at 20:16
benefits to both methods! Thanks again! – blazetopher Jun 21 '13 at 20:17
@blazetopher have a look at my updated solution. – Ashwini Chaudhary Jun 21 '13 at 20:20
Intriguing that your timeit results are different than @T.J's, I'm editing my post to show my comparison – blazetopher Jun 21 '13 at 20:30
@blazetopher note that I've modified FJ's solution to handle strings as well. – Ashwini Chaudhary Jun 21 '13 at 20:32
@AshwaniChaudhary - yes, thank you. When I run the timeit comparison, the comprehension comes out slightly faster for me for whatever reason – blazetopher Jun 21 '13 at 20:37

Andrew Clark · Accepted Answer · 2013-06-21T20:34:38.063

1

Here is a pretty straightforward approach that uses a list comprehension:

>>> data = [[0, 1, 3, 7, 8, 11, 12], [8, 0, 1, 2, 3, 14], 2]
>>> [a for x in data for a in (x if isinstance(x, list) else [x])]
[0, 1, 3, 7, 8, 11, 12, 8, 0, 1, 2, 3, 14, 2]

Here are timing comparisons, it looks like my version is slightly faster (note that I modified my code to use collections.Iterable as well to make sure the comparison is fair):

In [9]: %timeit list(chain.from_iterable(solve(data)))
100000 loops, best of 3: 9.22 us per loop

In [10]: %timeit [a for x in data for a in (x if isinstance(x, Iterable) else [x])]
100000 loops, best of 3: 6.45 us per loop

edited Jun 21 '13 at 20:34

answered Jun 21 '13 at 19:59

Andrew Clark

202,379
35
273
306

Thanks! I was very close to this myself (not posted) but wasn't getting the logic portion correct. Do you know if this is more/less performant than @Ashwini's response? – blazetopher Jun 21 '13 at 20:06
@blazetopher Just added a timing comparison. – Andrew Clark Jun 21 '13 at 20:14
1

Don't use `input` as a variable name. – Ashwini Chaudhary Jun 21 '13 at 20:22
@AshwiniChaudhary Good point, thanks! – Andrew Clark Jun 21 '13 at 20:34
@AshwiniChaudhary Agreed, silly oversight on my part – blazetopher Jun 21 '13 at 20:36

oleg · Answer 3 · 2013-06-21T20:11:49.213

You can Use generator to make list consist of iterables only

((i if isinstance(i,Iterable) else [i])

And then use any of following methods to concatenate them into one list:

would You try sum

from collections import Iterable
lis = [[0, 1, 3, 7, 8, 11, 12], [8, 0, 1, 2, 3, 14], 2]
sum(((i if isinstance(i,Iterable) else [i]) for i in lis), [])

You should provide initial value for sum [] to start sum from

or itertools.chain() but in this case You should unpack Your upper-level list into set of lists

import itertools
lis = [[0, 1, 3, 7, 8, 11, 12], [8, 0, 1, 2, 3, 14], 2]
list(itertools.chain(*((i if isinstance(i,Iterable) else [i]) for i in lis)))

or just list comprehension

[a for x in input for a in (x if isinstance(x, Iterable) else [x])]

But One should mind that strings are iterable as well. And If there will be string on upper level it would be spited into chars.

Your methods will work only if all the items of `lis` are iterable. — Ashwini Chaudhary, Jun 21 '13 at 20:01

Concatenation of inner lists or ints

3 Answers3

Linked