How can I use list comprehensions to process a nested list?

Question

I have this nested list:

l = [['40', '20', '10', '30'], ['20', '20', '20', '20', '20', '30', '20'], ['30', '20', '30', '50', '10', '30', '20', '20', '20'], ['100', '100'], ['100', '100', '100', '100', '100'], ['100', '100', '100', '100']]

I want to convert each element in l to float. I have this code:

newList = []
for x in l:
    for y in x:
        newList.append(float(y))

How can I solve the problem with a nested list comprehension instead?

_{See also: How can I get a flat result from a list comprehension instead of a nested list?}

@GregHewgill: OP didn't reply, but based on the answer they accepted, seems they wanted to keep the nesting as is. — smci, Feb 02 '18 at 20:39
See also: https://treyhunner.com/2015/12/python-list-comprehensions-now-in-color/ — Karl Knechtel, Feb 18 '22 at 21:32
`[float(y) for y in x for x in l]` **does not work by itself**. It is only working because of the existing definition of `x` left over from previous code. — Karl Knechtel, Aug 02 '22 at 21:43

score 459 · Accepted Answer · edited Jan 26 '23 at 17:52

459

Here is how you would do this with a nested list comprehension:

[[float(y) for y in x] for x in l]

This would give you a list of lists, similar to what you started with except with floats instead of strings.

If you want one flat list, then you would use

[float(y) for x in l for y in x]

Note the loop order - for x in l comes first in this one.

edited Jan 26 '23 at 17:52

user2357112

260,549
28
431
505

answered Aug 06 '13 at 06:05

Andrew Clark

202,379
35
273
306

Rahul · Answer 2 · 2022-10-21T05:50:31.907

361

Here is how to convert nested for loop to nested list comprehension:

Here is how nested list comprehension works:

            l a b c d e f
            ↓ ↓ ↓ ↓ ↓ ↓ ↓
In [1]: l = [ [ [ [ [ [ 1 ] ] ] ] ] ]
In [2]: for a in l:
   ...:     for b in a:
   ...:         for c in b:
   ...:             for d in c:
   ...:                 for e in d:
   ...:                     for f in e:
   ...:                         print(float(f))
   ...:                         
1.0

In [3]: [float(f)
         for a in l
   ...:     for b in a
   ...:         for c in b
   ...:             for d in c
   ...:                 for e in d
   ...:                     for f in e]
Out[3]: [1.0]

For your case, if you want a flat list, it will be something like this.

In [4]: new_list = [float(y) for x in l for y in x]

edited Oct 21 '22 at 05:50

answered Jul 13 '17 at 11:27

Rahul

10,830
4
53
88

35

Super useful! Makes it clear that loops (top-to-bottom) are ordered left to right in the generator. This not obvious since in `(f(x) for x in l)` places the second line of the for-loop equivalent on the left. – user48956 Jan 11 '18 at 18:11
@user48956 yeah, i don't think it's very intuitive to have a nested list comprehension as a one liner. such usage would be an antipattern imo – dtc Apr 14 '21 at 23:19

score 62 · Answer 3 · answered Mar 26 '17 at 02:49

62

Not sure what your desired output is, but if you're using list comprehension, the order follows the order of nested loops, which you have backwards. So I got the what I think you want with:

[float(y) for x in l for y in x]

The principle is: use the same order you'd use in writing it out as nested for loops.

answered Mar 26 '17 at 02:49

Harry Binswanger

1,069
10
12

this should be the answer, as some times we don't want to square bracket the iteratool – zinking Apr 19 '17 at 01:48
3

this may not be the correct answer as it outputs a non nested list, but it is what I was looking for, specially *the principle*. Thanks! – Rodrigo E. Principe Sep 29 '17 at 10:52
2

This is not correct: should have bracket around `[float(y)]` – WestCoastProjects Mar 05 '21 at 15:36

score 55 · Answer 4 · answered Aug 06 '13 at 06:04

>>> l = [['40', '20', '10', '30'], ['20', '20', '20', '20', '20', '30', '20'], ['30', '20', '30', '50', '10', '30', '20', '20', '20'], ['100', '100'], ['100', '100', '100', '100', '100'], ['100', '100', '100', '100']]
>>> new_list = [float(x) for xs in l for x in xs]
>>> new_list
[40.0, 20.0, 10.0, 30.0, 20.0, 20.0, 20.0, 20.0, 20.0, 30.0, 20.0, 30.0, 20.0, 30.0, 50.0, 10.0, 30.0, 20.0, 20.0, 20.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0]

score 11 · Answer 5 · edited Feb 07 '22 at 01:16

I wanted to share how the list comprehension actually works, especially for nested list comprehensions:

new_list= [float(x) for x in l]

is actually the same as:

new_list=[]
for x in l:
    new_list.append(float(x))

And now for the nested list comprehension:

[[float(y) for y in x] for x in l]

is the same as:

new_list=[]
for x in l:
    sub_list=[]
    for y in x:
        sub_list.append(float(y))

    new_list.append(sub_list)

print(new_list)

output:

[[40.0, 20.0, 10.0, 30.0], [20.0, 20.0, 20.0, 20.0, 20.0, 30.0, 20.0], [30.0, 20.0, 30.0, 50.0, 10.0, 30.0, 20.0, 20.0, 20.0], [100.0, 100.0], [100.0, 100.0, 100.0, 100.0, 100.0], [100.0, 100.0, 100.0, 100.0]]

score 9 · Answer 6 · answered Jan 18 '18 at 11:45

I had a similar problem to solve so I came across this question. I did a performance comparison of Andrew Clark's and narayan's answer which I would like to share.

The primary difference between two answers is how they iterate over inner lists. One of them uses builtin map, while other is using list comprehension. Map function has slight performance advantage to its equivalent list comprehension if it doesn't require the use lambdas. So in context of this question map should perform slightly better than list comprehension.

Lets do a performance benchmark to see if it is actually true. I used python version 3.5.0 to perform all these tests. In first set of tests I would like to keep elements per list to be 10 and vary number of lists from 10-100,000

>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,10))]*10]"
>>> 100000 loops, best of 3: 15.2 usec per loop   
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,10))]*10]"
>>> 10000 loops, best of 3: 19.6 usec per loop 

>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,10))]*100]"
>>> 100000 loops, best of 3: 15.2 usec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,10))]*100]"
>>> 10000 loops, best of 3: 19.6 usec per loop 

>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,10))]*1000]"
>>> 1000 loops, best of 3: 1.43 msec per loop   
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,10))]*1000]"
>>> 100 loops, best of 3: 1.91 msec per loop

>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,10))]*10000]"
>>> 100 loops, best of 3: 13.6 msec per loop   
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,10))]*10000]"
>>> 10 loops, best of 3: 19.1 msec per loop

>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,10))]*100000]"
>>> 10 loops, best of 3: 164 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,10))]*100000]"
>>> 10 loops, best of 3: 216 msec per loop

In the next set of tests I would like to raise number of elements per lists to 100.

>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,100))]*10]"
>>> 10000 loops, best of 3: 110 usec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,100))]*10]"
>>> 10000 loops, best of 3: 151 usec per loop

>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,100))]*100]"
>>> 1000 loops, best of 3: 1.11 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,100))]*100]"
>>> 1000 loops, best of 3: 1.5 msec per loop

>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,100))]*1000]"
>>> 100 loops, best of 3: 11.2 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,100))]*1000]"
>>> 100 loops, best of 3: 16.7 msec per loop

>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,100))]*10000]"
>>> 10 loops, best of 3: 134 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,100))]*10000]"
>>> 10 loops, best of 3: 171 msec per loop

>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,100))]*100000]"
>>> 10 loops, best of 3: 1.32 sec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,100))]*100000]"
>>> 10 loops, best of 3: 1.7 sec per loop

Lets take a brave step and modify the number of elements in lists to be 1000

>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,1000))]*10]"
>>> 1000 loops, best of 3: 800 usec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,1000))]*10]"
>>> 1000 loops, best of 3: 1.16 msec per loop

>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,1000))]*100]"
>>> 100 loops, best of 3: 8.26 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,1000))]*100]"
>>> 100 loops, best of 3: 11.7 msec per loop

>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,1000))]*1000]"
>>> 10 loops, best of 3: 83.8 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,1000))]*1000]"
>>> 10 loops, best of 3: 118 msec per loop

>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,1000))]*10000]"
>>> 10 loops, best of 3: 868 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,1000))]*10000]"
>>> 10 loops, best of 3: 1.23 sec per loop

>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,1000))]*100000]"
>>> 10 loops, best of 3: 9.2 sec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,1000))]*100000]"
>>> 10 loops, best of 3: 12.7 sec per loop

From these test we can conclude that map has a performance benefit over list comprehension in this case. This is also applicable if you are trying to cast to either int or str. For small number of lists with less elements per list, the difference is negligible. For larger lists with more elements per list one might like to use map instead of list comprehension, but it totally depends on application needs.

However I personally find list comprehension to be more readable and idiomatic than map. It is a de-facto standard in python. Usually people are more proficient and comfortable(specially beginner) in using list comprehension than map.

score 3 · Answer 7 · edited Jan 17 '16 at 07:10

3

If you don't like nested list comprehensions, you can make use of the map function as well,

>>> from pprint import pprint

>>> l = l = [['40', '20', '10', '30'], ['20', '20', '20', '20', '20', '30', '20'], ['30', '20', '30', '50', '10', '30', '20', '20', '20'], ['100', '100'], ['100', '100', '100', '100', '100'], ['100', '100', '100', '100']] 

>>> pprint(l)
[['40', '20', '10', '30'],
['20', '20', '20', '20', '20', '30', '20'],
['30', '20', '30', '50', '10', '30', '20', '20', '20'],
['100', '100'],
['100', '100', '100', '100', '100'],
['100', '100', '100', '100']]

>>> float_l = [map(float, nested_list) for nested_list in l]

>>> pprint(float_l)
[[40.0, 20.0, 10.0, 30.0],
[20.0, 20.0, 20.0, 20.0, 20.0, 30.0, 20.0],
[30.0, 20.0, 30.0, 50.0, 10.0, 30.0, 20.0, 20.0, 20.0],
[100.0, 100.0],
[100.0, 100.0, 100.0, 100.0, 100.0],
[100.0, 100.0, 100.0, 100.0]]

edited Jan 17 '16 at 07:10

Remi Guan

21,506
17
64
87

answered Nov 27 '15 at 05:55

narayan

1,119
13
20

Your code generates map objects instead of lists: `>>> float_l = [map(float, nested_list) for nested_list in l]` `[[], [], [], [], [], []] ` but adding an additional call to list it works as expected: `>>> float_l = [list(map(float, nested_list)) for nested_list in l]` – pixelperfect Mar 17 '17 at 15:49
@pixelperfect that is due to the (*misinformed* ..) change in `python3` to return generators out of comprehensions. – WestCoastProjects Mar 04 '18 at 04:56

score 3 · Answer 8 · edited Apr 16 '18 at 09:12

3

This Problem can be solved without using for loop.Single line code will be sufficient for this. Using Nested Map with lambda function will also works here.

l = [['40', '20', '10', '30'], ['20', '20', '20', '20', '20', '30', '20'], ['30', '20', '30', '50', '10', '30', '20', '20', '20'], ['100', '100'], ['100', '100', '100', '100', '100'], ['100', '100', '100', '100']]

map(lambda x:map(lambda y:float(y),x),l)

And Output List would be as follows:

[[40.0, 20.0, 10.0, 30.0], [20.0, 20.0, 20.0, 20.0, 20.0, 30.0, 20.0], [30.0, 20.0, 30.0, 50.0, 10.0, 30.0, 20.0, 20.0, 20.0], [100.0, 100.0], [100.0, 100.0, 100.0, 100.0, 100.0], [100.0, 100.0, 100.0, 100.0]]

edited Apr 16 '18 at 09:12

Mikhail_Sam

10,602
11
66
102

answered Apr 27 '17 at 06:52

Aakash Goel

982
7
12

1

Do lambdas has any performance benefits over say @Andrew Clark or Harry Binswanger's solutions (the more vanilla list comprehension)? As lambdas seem harder to read. – Stefan Collier Oct 02 '17 at 13:45
1

In nearly any other general programming language we would use chained `map` - along the lines of what you show. But look how ugly it is in python - especially given the output above is not sufficient: it is a generator not a list. Need to add even _more_ boilerplate `list(list( .. ) )` to complete the picture. Try out `fluentpy` and similar libraries to escape this trap to some extent. – WestCoastProjects Mar 05 '21 at 15:31

score 2 · Answer 9 · answered Aug 06 '13 at 06:06

2

Yes, you can do it with such a code:

l = [[float(y) for y in x] for x in l]

answered Aug 06 '13 at 06:06

Victor

65
2
7

`[float(y) for y in x for x in l]` this would result to a bunch of 100's with a sum of 2400. – Boy Pasmo Aug 06 '13 at 06:09

score 2 · Answer 10 · answered Dec 28 '21 at 18:55

2

In case a flattened list is needed:

[y for x in l for y in x]

In case a nested list (list in list) is needed:

[[float(y) for y in x] for x in l]

answered Dec 28 '21 at 18:55

learner

603
1
3
15

score 0 · Answer 11 · answered Jul 12 '16 at 10:46

0

The best way to do this in my opinion is to use python's itertools package.

>>>import itertools
>>>l1 = [1,2,3]
>>>l2 = [10,20,30]
>>>[l*2 for l in itertools.chain(*[l1,l2])]
[2, 4, 6, 20, 40, 60]

answered Jul 12 '16 at 10:46

Thomasillo

43
4

score 0 · Answer 12 · edited Mar 22 '18 at 14:59

0

    deck = [] 
    for rank in ranks:
        for suit in suits:
            deck.append(('%s%s')%(rank, suit))

This can be achieved using list comprehension:

[deck.append((rank,suit)) for suit in suits for rank in ranks ]

edited Mar 22 '18 at 14:59

kutschkem

7,826
3
21
56

answered Mar 22 '18 at 11:43

ADITYA KUMAR

57
5

1

This does not appear to address the question on top at all. Please note that everything posted as an answer must be an attempt to answer the question it is posted to. – Baum mit Augen Mar 22 '18 at 11:46
While this code snippet may solve the question, [including an explanation](//meta.stackexchange.com/questions/114762/explaining-entirely-code-based-answers) really helps to improve the quality of your post. Remember that you are answering the question for readers in the future, and those people might not know the reasons for your code suggestion. Please also try not to crowd your code with explanatory comments, this reduces the readability of both the code and the explanations! – Filnor Mar 22 '18 at 11:46
Nested for loop using list comprehension, – ADITYA KUMAR Mar 22 '18 at 11:56
1

Ok, so apparently, this is an attempt to answer the question. However, this appears to be about an entirely different scenario than in OP, you don't even deal with nested lists as input, and even if you change that your suggestion is pretty much what OP already tried. Also, I don't see how an example about cards helps when the question is about converting strings to float. – Baum mit Augen Mar 22 '18 at 12:02
ignore above... your comment helped me figure this out massively. Thanks – BeerHuntor Oct 12 '21 at 18:27

score 0 · Answer 13 · answered Sep 06 '18 at 21:46

0

Yes you can do the following.

[[float(y) for y in x] for x in l]

answered Sep 06 '18 at 21:46

user1142317

533
1
8
20

How can I use list comprehensions to process a nested list?

13 Answers13

Linked

Related