What's the recommended way to flatten nested lists since the deprecation of the compiler package?
>>> from compiler.ast import flatten
>>> flatten(["junk",["nested stuff"],[],[[]]])
['junk', 'nested stuff']
I know that there are a few stack overflow answers for list flattening, but I'm hoping for the pythonic, standard package, "one, and preferably only one, obvious way" to do this.
itertools.chain is the best solution for flattening any nested iterable one level - it's highly efficient compared to any pure-python solution.
That said, it will work on all iterables, so some checking is required if you want to avoid it flattening out strings, for example.
Likewise, it won't magically flatten out to an arbitrary depth. That said, generally, such a generic solution isn't required - instead it's best to keep your data structured so that it doesn't require flattening in that way.
Edit: I would argue that if one had to do arbitrary flattening, this is the best way:
import collections
def flatten(iterable):
for el in iterable:
if isinstance(el, collections.Iterable) and not isinstance(el, str):
yield from flatten(el)
else:
yield el
Remember to use basestring in 2.x over str, and for subel in flatten(el): yield el instead of yield from flatten(el) pre-3.3.
As noted in the comments, I would argue this is the nuclear option, and is likely to cause more problems than it solves. Instead, the best idea is to make your output more regular (output that contains one item still give it as a one item tuple, for example), and do regular flattening by one level where it is introduced, rather than all at the end.
This will produce more logical, readable, and easier to work with code. Naturally, there are cases where you need to do this kind of flattening (if the data is coming from somewhere you can't mess with, so you have no option but to take it in the poorly-structured format), in which case, this kind of solution might be needed, but in general, it's probably a bad idea.
Your stated function takes a nested list and flattens that into a new list.
To flatten an arbitrarily nested list into a new list, this works on Python 3 as you expect:
import collections
def flatten(x):
result = []
for el in x:
if isinstance(x, collections.Iterable) and not isinstance(el, str):
result.extend(flatten(el))
else:
result.append(el)
return result
print(flatten(["junk",["nested stuff"],[],[[]]]))
Prints:
['junk', 'nested stuff']
If you want a generator that does the same thing:
def flat_gen(x):
def iselement(e):
return not(isinstance(e, collections.Iterable) and not isinstance(e, str))
for el in x:
if iselement(el):
yield el
else:
for sub in flat_gen(el): yield sub
print(list(flat_gen(["junk",["nested stuff"],[],[[[],['deep']]]])))
# ['junk', 'nested stuff', 'deep']
For Python 3.3 and later, use yield from instead of the loop:
def flat_gen(x):
def iselement(e):
return not(isinstance(e, collections.Iterable) and not isinstance(e, str))
for el in x:
if iselement(el):
yield el
else:
yield from flat_gen(el)
You can use flatten function from funcy library:
from funcy import flatten, isa
flat_list = flatten(your_list)
You can also explicitly specify which values to follow:
# Follow only sets
flat_list = flatten(your_list, follow=isa(set))
Take a peek at its implementation if you want an algorythm.
There's no built-in method for an list with arbitrary nesting, but something like this...
def flatten(l):
for i in l:
if isinstance(i, (list, tuple)):
for ii in flatten(i):
yield ii
else:
yield i
>>> l = ["junk",["nested stuff"],[],[[]]]
>>> list(flatten(l))
['junk', 'nested stuff']
...will work for lists and tuples. If you want to support any object which you could use in a for item in object expression, then it's probably best to using duck-typing like this...
def flatten(l):
for i in l:
if isinstance(i, (str, bytes)):
yield i
else:
try:
for ii in flatten(i):
yield ii
except TypeError:
yield i
>>> l = ["junk",["nested stuff"],[],[[]]]
>>> list(flatten(l))
['junk', 'nested stuff']
...which is slightly more robust than checking isinstance(el, Iterable), since it won't cope with some cases, such as this...
class Range10:
def __getitem__(self, index):
if index >= 10:
raise IndexError
return index
>>> import collections
>>> r10 = Range10()
>>> isinstance(r10, collections.Iterable)
False
>>> list(Range10())
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
My ugly while-chain solution, just for fun:
from collections import Iterable
from itertools import chain
def flatten3(seq, exclude=(str,)):
sub = iter(seq)
try:
while sub:
while True:
j = next(sub)
if not isinstance(j, Iterable) or isinstance(j, exclude):
yield j
else:
sub = chain(j, sub)
break
except StopIteration:
return
Built for recursion: Python 3.6
def flatten(lst):
"""Flattens a list of lists"""
return [subelem for elem in lst
for subelem in elem]
Define your types in a list and use the any builtin to check
Just another recursion: Python 3.6 . Works fine for complex nested list of lists/strings/digits and combination of all:
def list_flattener(nestedList):
nonListElems=[]
listElems=[]
nestedListCounter=0
for nc in range(len(nestedList)):
if type(nestedList[nc])==list:
nestedListCounter+=1
listElems=nestedList[nc]+listElems
else:nonListElems.append(nestedList[nc])
if nestedListCounter==0: return (nestedList)
else:
nestedList=listElems+nonListElems
return list_flattener(nestedList)
See the example below :
>>> nestedList=['arash',[[1,'anotherOne',[12,'stringX',['kave']]], 'stringY']]
>>> list_flattener(nestedList)
['kave', 12, 'stringX', 1, 'anotherOne', 'stringY', 'arash']
