numpy - column-wise and row-wise sums of a given 2d matrix

Question

I have this numpy matrix (ndarray).

    array([[ 1,  2,  3,  4,  5],
           [ 6,  7,  8,  9, 10],
           [11, 12, 13, 14, 15],
           [16, 17, 18, 19, 20],
           [21, 22, 23, 24, 25]])

I want to calculate the column-wise and row-wise sums.

I know this is done by calling respectively

np.sum(mat, axis=0)   ### column-wise sums

np.sum(mat, axis=1)   ### row-wise sums   

but I cannot understand these two calls.

Why is axis 0 giving me the sums column-by-column?!
Shouldn't it be the other way around?

I thought the rows are axis 0, and the columns are axis 1.

What I am seeing as a behavior here looks counter-intuitive
(but I am sure it's OK, I guess I am just missing something important).

I am just looking for some intuitive explanation here.

Thanks in advance.

Answer 1

Intuition around arrays and axes

I want to offer 3 types of intuitions here.

1. Graphical (How to imagine them visually)
2. Physical (How they are physically stored)
3. Logical (How to work with them logically)

---

Graphical intuition

Consider a numpy array as a n-dimensional object. This n-dimensional object contains elements in each of the directions as below.

Axes in this representation are the direction of the tensor. So, a 2D matrix has only 2 axes, while a 4D tensor has 4 axes.

Sum in a given axis can be essentially considered as a reduction in that direction. Imagine a 3D tensor being squashed in such a way that it becomes flat (a 2D tensor). The axis tells us which direction to squash or reduce it in.

---

Physical intuition

Numpy stores its ndarrays as contiguous blocks of memory. Each element is stored in a sequential manner every n bytes after the previous.

(images referenced from this excellent SO post)

So if your 3D array looks like this -

Then in memory its stores as -

When retrieving an element (or a block of elements), NumPy calculates how many strides (bytes) it needs to traverse to get the next element in that direction/axis. So, for the above example, for axis=2 it has to traverse 8 bytes (depending on the datatype) but for axis=1 it has to traverse 8*4 bytes, and axis=0 it needs 8*8 bytes.

Axes in this representation is basically the series of next elements after a given stride. Consider the following array -

print(X)
print(X.strides)

[[0 2 1 4 0 0 0]
 [5 0 0 0 0 0 0]
 [8 0 0 0 0 0 0]
 [0 0 0 0 0 0 0]
 [0 0 1 0 0 0 0]
 [0 0 0 1 0 0 0]]

#Strides (bytes) required to traverse in each axis.
(56, 8)

In the above array, every element after 56 bytes from any element is the next element in axis=0 and every element after 8 bytes from any element is in axis=1. (except from the last element)

Sum or reduction in this regards means taking a sum of every element in that strided series. So, sum over axis=0 means that I need to sum [0,5,8,0,0,0], [2,0,0,0,0,0], ... and sum over axis=1 means just summing [0 2 1 4 0 0 0] , [5 0 0 0 0 0 0], ...

---

Logical intuition

This interpretation has to do with element groupings. A numpy stores its ndarrays as groups of groups of groups ... of elements. Elements are grouped together and contain the last axis (axis=-1). Then another grouping over them creates another axis before it (axis=-2). The final outermost group is the axis=0.

These are 3 groups of 2 groups of 5 elements.

Similarly, the shape of a NumPy array is also determined by the same.

1D_array = [1,2,3]
2D_array = [[1,2,3]]
3D_array = [[[1,2,3]]]
...

Axes in this representation are the group in which elements are stored. The outermost group is axis=0 and the innermost group is axis=-1.

Sum or reduction in this regard means that I reducing elements across that specific group or axis. So, sum over axis=-1 means I sum over the innermost groups. Consider a (6, 5, 8) dimensional tensor. When I say I want a sum over some axis, I want to reduce the elements lying in that grouping / direction to a single value that is equal to their sum.

So,

• np.sum(arr, axis=-1) will reduce the inner most groups (of length 8) into a single value and return (6,5,1) or (6,5).
• np.sum(arr, axis=-2) will reduce the elements that lie in the 1st axis (or -2nd axis) direction and reduce those to a single value returning (6,1,8) or (6,8)
• np.sum(arr, axis=0) will similarly reduce the tensor to (1,5,8) or (5,8).

Hope these 3 intuitions are beneficial to anyone trying to understand how axes and NumPy tensors work in general and how to build an intuitive understanding to work better with them.

Answer 2

Let's start with a one dimensional example:

a, b, c, d, e = 0, 1, 2, 3, 4
arr = np.array([a, b, c, d, e])

If you do,

arr.sum(0) 

Output

10

That is the sum of the elements of the array

a + b + c + d + e

Now before moving on a 2 dimensional example. Let's clarify that in numpy the sum of two 1 dimensional arrays is done element wise, for example:

a = np.array([1, 2, 3, 4, 5])
b = np.array([6, 7, 8, 9, 10])
print(a + b)

Output

[ 7  9 11 13 15]

Now if we change our initial variables to arrays, instead of scalars, to create a two dimensional array and do the sum

a = np.array([1, 2, 3, 4, 5])
b = np.array([6, 7, 8, 9, 10])
c = np.array([11, 12, 13, 14, 15])
d = np.array([16, 17, 18, 19, 20])
e = np.array([21, 22, 23, 24, 25])

arr = np.array([a, b, c, d, e])
print(arr.sum(0))

Output

[55 60 65 70 75]

The output is the same as for the 1 dimensional example, i.e. the sum of the elements of the array:

a + b + c + d + e

Just that now the elements of the arrays are 1 dimensional arrays and the sum of those elements is applied. Now before explaining the results, for axis = 1, let's consider an alternative notation to the notation across axis = 0, basically:

np.array([arr[0, :], arr[1, :], arr[2, :], arr[3, :], arr[4, :]]).sum(0) # [55 60 65 70 75]

That is we took full slices in all other indices that were not the first dimension. If we swap to:

res = np.array([arr[:, 0], arr[:, 1], arr[:, 2], arr[:, 3], arr[:, 4]]).sum(0)
print(res)

Output

[ 15  40  65  90 115]

We get the result of the sum along axis=1. So to sum it up you are always summing elements of the array. The axis will indicate how this elements are constructed.

Answer 3

Intuitively, 'axis 0' goes from top to bottom and 'axis 1' goes from left to right. Therefore, when you sum along 'axis 0' you get the column sum, and along 'axis 1' you get the row sum.

As you go along 'axis 0', the row number increases. As you go along 'axis 1' the column number increases.

Answer 4

Think of a 1-dimension array:

mat=array([ 1,  2,  3,  4,  5])

Its items are called by mat[0], mat[1], etc

If you do:

np.sum(mat, axis=0) 

it will return 15

In the background, it sums all items with mat[0], mat[1], mat[2], mat[3], mat[4]

meaning the first index (axis=0)

Now consider a 2-D array:

mat=array([[ 1,  2,  3,  4,  5],
           [ 6,  7,  8,  9, 10],
           [11, 12, 13, 14, 15],
           [16, 17, 18, 19, 20],
           [21, 22, 23, 24, 25]])

When you ask for

np.sum(mat, axis=0)

it will again sum all items based on the first index (axis=0) keeping all the rest same. This means that

mat[0][1], mat[1][1], mat[2][1], mat[3][1], mat[4][1]

will give one sum

mat[0][2], mat[1][2], mat[2][2], mat[3][2], mat[4][2]

will give another one, etc

If you consider a 3-D array, the logic will be the same. Every sum will be calculated on the same axis (index) keeping all the rest same. Sums on axis=0 will be produced by:

mat[0][1][1],mat[1][1][1],mat[2][1][1],mat[3][1][1],mat[4][1][1]

etc

Sums on axis=2 will be produced by:

mat[2][3][0], mat[2][3][1], mat[2][3][2], mat[2][3][3], mat[2][3][4]

etc

I hope you understand the logic. To keep things simple in your mind, consider axis=position of index in a chain index, eg axis=3 on a 7-mensional array will be:

mat[0][0][0][this is our axis][0][0][0]