3

The shape of a numpy array is changing when performing specific slicing in a somewhat unexpected manner

I have tried several ways of slicing the same array but slight differences lead to different outcomes in the shape of the array 

import numpy as np
z = np.zeros((1,9,10,2))

# This makes sense
print(z[...,[1,0]].shape)
# (1, 9, 10, 2)
print(z[0,...].shape)
# (9, 10, 2)
print(z[0:1,...,[1,0]].shape)
# (1, 9, 10, 2)
print(z[0][...,[1,0]].shape)
# (9, 10, 2)

# This doesn't, I would expect (9, 10, 2) in both cases
print(z[0,:,:,[1,0]].shape)
# (2, 9, 10)
print(z[0,...,[1,0]].shape)
# (2, 9, 10)

In the last two examples I do not understand why the last axis is moved to the first position.

I am using Python 3.6.4 with numpy 1.15.1

Jose Javier Gonzalez Ortiz
  • 468
  • 1
  • 5
  • 16
  • This is a case of mixed basic and advanced indexing (see the indexing docs page). When a slice (or ellipsis) is in the middle of other indexes, it is move to the end of the result. There are duplicate SO as well. You already found ways around it. – hpaulj May 24 '19 at 21:03
  • https://stackoverflow.com/q/55829631/901925 – hpaulj May 24 '19 at 21:11

1 Answers1

2

The reason why you might find the result in the two last cases unexpected, is because the indexing of the array is following the rules of advanced indexing, even though you're also indexing with slices.

For an extensive explanation behind this behaviour, you can check combining advanced and basic indexing. In these last cases in which you're getting unexpected resulting shapes. In the docs, you'll see that one of the mentioned scenarios in which we might obtain unexpected results is when:

  • The advanced indexes are separated by a slice, Ellipsis or newaxis. For example x[arr1, :, arr2].

In your case, although you're only using an integer for indexing along the first axis, it is broadcast and both arrays are iterated as one. In this case the dimensions resulting from the advanced indexing operation come first in the result array, and the sliced dimensions after that.

The key here is to understand that as mentioned in the docs, it is like concatenating the indexing result for each advanced index element.

So in essence it is doing the same as:

z = np.random.random((1,9,10,2))
a = np.concatenate([z[0,:,:,[1]], z[0,:,:,[0]]], axis=0)

Which is the same as the last indexing operation:

b = z[0,:,:,[1,0]]
np.allclose(a,b)
# True

What is the reason behind this behaviour?

A general rule to keep in mind is that:

The resulting axes introduced by the arrays indexes are at the front, unless they are consecutive.

So since the indexing arrays here are not consecutive, the resulting axes on which they’ve been used will come at the front, and the sliced dimension at the back.

While it might seem very weird when indexing with 1-dimensional arrays, take into account that it is also possible to index with arrays of an arbitrary amount of dimensions. Say we are indexing the same example array both on the first and last axes with 3d arrays, both say with shape (3,4,2). So we know that the final array will somewhere also have the shape (3,4,2), since both indexing arrays broadcast to the same shape. Now the question is, where should the full slice taken between the indexing arrays be placed?

Given that it is no longer as clear that it should go in the middle, there is a convention in these cases which is that sliced dimensions go at the end. So in such cases it will be our task to rearrange the dimensions of the array to match our expected output. On the example above what we could do is to swap the last two axes and get as we expected with using swapaxes to get the dimensions arranged as expected.

yatu
  • 86,083
  • 12
  • 84
  • 139