7

PyTorch doesn't seem to have documentation for tensor.stride(). Can someone confirm my understanding?

My questions are three-fold.

  1. Stride is for accessing an element in the storage. So stride size will be the same as the dimension of the tensor. Correct?

  2. For each dimension, the corresponding element of stride tells how much it takes to move along the 1-dimensional storage. Correct?

For example:

In [15]: x = torch.arange(1,25)

In [16]: x
Out[16]:
tensor([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17, 18
, 19, 20, 21, 22, 23, 24])

In [17]: a = x.view(4,3,2)

In [18]: a
Out[18]:
tensor([[[ 1,  2],
         [ 3,  4],
         [ 5,  6]],

        [[ 7,  8],
         [ 9, 10],
         [11, 12]],

        [[13, 14],
         [15, 16],
         [17, 18]],

        [[19, 20],
         [21, 22],
         [23, 24]]])

In [20]: a.stride()
Out[20]: (6, 2, 1)
  1. How does having this information help perform tensor operations efficiently? Basically this is showing the memory layout. So how does it help?
aerin
  • 20,607
  • 28
  • 102
  • 140
  • 1
    Assuming `tensor` is doing the same as `numpy`, `strides` allows the underlying `c` code to step through the 1d databuffer (memory) in an dimension with nearly the same ease. It can iterate through the 'blocks' by moving forward 6 elements at a time, through 'rows' by 2 at a time, etc. It's `shape` and `strides` that make `views` possible. – hpaulj Jul 25 '20 at 22:05
  • Consider that `x = torch.tensor([1, 2, 3, 4])` has `x.stride()` of `1` but if you do `y = x[::2]` then `y.stride()` will be `2`. Basically what happens here is that we avoided creating a copy of `y` and instead `y` is a view of `x`, i.e. it shares the same underlying memory, just uses a different stride to access it. – jodag Jul 26 '20 at 01:21
  • https://discuss.pytorch.org/t/pytorch-tensor-stride-how-it-works/90537/2?u=purushothaman_srikan – Purushothaman Srikanth Feb 27 '21 at 14:26

0 Answers0