Python list comprehension

Question

Possible Duplicate:
List comprehension for running total

I'm trying to write a concise list comprehension statement to create a cdf: For example:

print f([0.2, 0.3,0.1,0.4])
[0.2,0.5,0.6,1.0]

A standard procedure would look like this (I want to write a list comprehension for the function f()):

def f(probabilities) :

    sum = 0
    returnList = []
    for count in probabilities:
        sum +=count
        returnList = returnList + [sum]
    return returnList

Edit: I found a function numpy.cumsum(). I'll check if it uses list comprehensions.

@Elalfer — It sounds like he wants to write a list comprehension whose behavior is identical to that of his `f()` function. — Ben Blank, Jan 25 '11 at 20:56
then you could tag the question also with numpy/ scipy. numpy.cumsum() does not use list comprehension. Could you elaborate more what you are trying to achieve? — eat, Jan 25 '11 at 21:47
@jleedev and others who closed this: I'm newbie here in SO, but I wouldn't consider this question similar enough to the possible duplicate. Specifically see the update of my answer, where it should be clear that the question is not (only) about running total. Essentially a cdf must sum up to 1 and as the OP identified interest to Numpy, why not let more answers to flow in ;-). — eat, Jan 25 '11 at 22:20

tokland · Accepted Answer · 2014-12-03T20:22:51.983

8

That operation is so common that many languages (mainly functional ones, but not only) provide abstractions for it, usually with the name scanl (it's like a reduce with intermediate results). Let's call it ireduce ("iterative reduce"):

def ireduce(f, state, it):
    for x in it:
        state = f(state, x)
        yield state

And now use it:

import operator

def f(probabilities):
    return ireduce(operator.add, 0, probabilities)

print(list(f([0.2, 0.3,0.1,0.4])))
# [0.2, 0.5, 0.6, 1.0]

edited Dec 03 '14 at 20:22

answered Jan 25 '11 at 21:10

tokland

66,169
13
144
170

This is a good thing to have in one's toolbox; but surely we can think of a more Pythonic name rather than copying what the other languages call it? FWIW, C++ calls this `std::partial_sum`, and uses addition as the operation by default. – Karl Knechtel Jan 25 '11 at 21:57
@Karl. Some people call it ireduce, I kind of like it. "partial_sum" is a good name when adding but with other operations seems somewhat misleading. – tokland Jan 25 '11 at 22:06

score 6 · Answer 2 · edited Jan 25 '11 at 20:57

6

[sum(probabilities[:i+1]) for i in range(len(probabilities))]

But don't do that because it's O(n^2). Python list comprehensions weren't designed for this. Use the procedural code that you already wrote.

edited Jan 25 '11 at 20:57

S.Lott

384,516
81
508
779

answered Jan 25 '11 at 20:56

novalis

1,112
6
12

Now that is one awesome list comprehension. – user225312 Jan 25 '11 at 21:02

score 1 · Answer 3 · answered Jan 25 '11 at 21:16

It's not really pretty, and it's not using list comprehensions, but you can do this with the reduce() function, where the accumulated value is a tuple holding the current sum and the result list:

a = [0.2, 0.3, 0.1, 0.4]
reduce((lambda result, val: (result[0] + val, result[1] + [result[0] + val])), a, (0, []))[1]

Python's lack of support for multi-line lambda's makes this kind of ugly. Using a separate function would be better:

    a = [0.2, 0.3, 0.1, 0.4]   
    def accumulate(result, val):
        return (result[0] + val, result[1] + [result[0] + val])

    reduce(accumulate, a, (0, []))[1]

Python list comprehension

3 Answers3