Partial column sum of a matrix or nested list

Question

The code listed below is an attempt to obtain a matrix with a partial column-sum of another, such that the rows, row[r], of the resultant matrix is the partial column-sum of the original matrix from row=0 to row=r.

For example, given

A = [[0,0,0],
     [4,5,6],
     [7,8,9],
     [10,11,12]]

I would like to obtain

B=[[0,0,0],
   [4,5,6],
   [11,13,15],
   [21,24,27]]

1. Is there an alternative that allow me to eliminate the for-loop in the following code and which allows me to use pure list-comprehension instead?

2. Do you consider list-comprehension will be more computational efficient than for-loops or map/lambda provided that the actual matrices I'm working on are relatively large?

My current code is below:

import numpy as np
# Define matrices M and S
M= np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
H = np.array([0.1, 0.2, 0.3])
# Define matrix S with: S[0] = [0,0,0,0] and S[r>0][c] = M[r][c]xH[r]
S = np.array([[x if r != 0 else 0 for x in [M[r][c] * H[r] for c in range(0, len(M[r]))]] for r in range(len(M))])
# initialize matrix L
L = np.array(np.zeros((int(len(M)),int(len(M[0])))))
#Update Matrix L: L[r][c] = Sum[S[0][c] to S[i=r-1][c]]
for r in range(0, len(L)):
        L[r] = [sum([row[i] for row in S[0:r+1]]) for i in range(0,len(S[0]))]
print("S", S)
print("L", L)

Answer 1

When using numpy, the fastest way to do something is usually to use broadcasting, or vectorization. You can read more about this here, but basically this means to apply a function to an array of data all at once, rather than by iterating over the array by using a for-loop.

This may have been what you were alluding to when you talked about list-comprehensions, but these at heart are just a different way of writing for-loops.

The function you are looking for in this case is np.cumsum(), using axis=0 to indicate that we want to cumulatively sum over each column. This is a vectorized method of applying the logic you seek, and will be a lot faster than your for-loop solution on large matrices.

>>> A = np.array([[0,0,0],[4,5,6],[7,8,9],[10,11,12]])
>>> np.cumsum(A, axis=0)
array([[ 0,  0,  0],
       [ 4,  5,  6],
       [11, 13, 15],
       [21, 24, 27]], dtype=int32)

Below is a graph showing a comparison of the speed of this approach and the approach in your question, as matrix size increases, which shows that the vectorized approach tends towards being approximately 10,000 times faster.

Code to generate:

import perfplot
import numpy as np

def cdjb_cumsum(A):
    return np.cumsum(A, axis=0)

def ajbg_forloop(A):
    L = np.array(np.zeros((int(len(A)),int(len(A[0])))))
    for r in range(0, len(L)):
        L[r] = [sum([row[i] for row in A[0:r+1]]) for i in range(0,len(A[0]))]
    return L

perfplot.show(
    setup=lambda n: np.random.rand(n, n),
    n_range=[2**k for k in range(20)],
    kernels=[
        cdjb_cumsum, ajbg_forloop
        ],
    xlabel='Size of Array (n*n)',
    )