0

I've been recently a bit confused about tensors. Say if we have a tensor with shape (3,2,3,4), we mean that in the first dimension there are 3 groups of numbers? OR it just means that there are exactly just 3 numbers in the first dimension?

Then here comes the second question, with the tensor A that has the shape (3,2), why is that the output of torch.max(A,0) returns a group of max values that contains 2 max values instead of 3,considering the fact that there are 3 numbers in the first dimension.

>>>a = torch.randn(3,2)
a tensor([[-1.1254, -0.1549],
        [-0.5308,  1.0427],
        [-0.1268,  1.0866]])
>>>torch.max(a,0)
torch.return_types.max(
values=tensor([-0.1268,  1.0866]),
indices=tensor([2, 2]))

I mean why doesn't it return a list of 3 max values?

Then the third question, if we have two tensors with shape(3,3,10,2) and (2,4,10,1), can we just concatenate these two tensors on the third dimension considering they have the same size on that dimension? If it is feasible, what is the reason behind it?

I'll be much appreciated if you help me understand this!

DominiqueNobody
  • 43
  • 10

2 Answers2

2

Tensors are just high level dimensions of vectors. You should start with a vector first. Ex: 4 elements vector A = [1,2,3,4]. A 6-vectors A together is B = [[1,2,3,4],[1,2,3,4],...,[1,2,3,4]]. This is called matrix (2 dimensions), the shape is 6x4. Now if you take 2 matrices B, it will form a tensor C, shape is (2,6,4). Extend C for a list of 3 C, you can get a 4-dimensions tensor (3,2,6,4) and so on. You can take this picture for better illustration.

illustration

For the torch max, torch.max(input, dim, keepdim=False, out=None) when you choose the dim=0 (the dimension to reduce to this mean you will find the max along the 0 axis (aka the first shape), it will follow through this axis to find the maximum number corresponding to the rest tensor (according to your tensor, you only have to check in the 2 elements vectors), which is left. If you extend this for higher dimension, like (3,4,5) shape, the max will be (4,5).

For the last question, there is a no for that tensor. Because when you look at the illustration figure, there is no way for you to just base on 1 shape to concatenate 2 different size matching tensor. You have to maintain the same shape for all dimensions except the on you are going to concatenate.

>>> b = np.random.randint(0,2,size=(2,3,2,3))
>>> a = np.random.randint(0,2,size=(2,3,1,3))
>>> np.concatenate([a,b],axis=2)

Else, the only way for you to do it is flattening both of the 2 tensors so that you only need a vector represent for both a and b.

Dishin H Goyani
  • 7,195
  • 3
  • 26
  • 37
dtlam26
  • 1,410
  • 11
  • 19
  • Much appreciated!!! Your explanation is just enlightening!!! Thank you!!! – DominiqueNobody Sep 29 '20 at 02:24
  • 2
    Your answer on the last question is actually incomplete. Concatenating two tensors is possible if they have the same values across all dimensions except the dimension they are concatenated at (e.g. two tensors with shape (1,2,3,4) and (1,2,5,4) can be concatenated on the third dimension). – Minh Nguyen Sep 29 '20 at 02:27
  • 1
    Yes, you are correct, my bad :) Good notice Minh – dtlam26 Sep 29 '20 at 02:36
  • 1
    @朱原昊 I have edited it again for you :D good luck – dtlam26 Sep 29 '20 at 02:47
0

About you first answer : think about images. that way it will be simple to visualize. Or in your example, x = np.zeros((3,2,3,4)) 3 number of 3d tensors having (2,3,4) dimensions each are stacked.

for second question you should have at least n-1 same dimensions to concatenate in n-th dimension. below picture will help to understand.

image : https://ibb.co/q5WnMD5

third question you mentioned max(a,0), where 0 represent axis, basically column and it is finding maximum values for each column.

Darshan
  • 71
  • 11
  • Thank you for your answer! Then if the tensor has the shape of (1,2,3,4) and $max(a,3)$ is used, what does it find? – DominiqueNobody Sep 29 '20 at 02:37
  • 1
    if you use `max(a,3)` then it corresponds axis=3 and in case of `shape=(1,2,3,4)` it will find maximum from each of column and result would be `shape= (1,2,3)`....if you use `max(a,2)` then it will find maximum out of each row and result would be `shape=(1,2,4)` – Darshan Sep 29 '20 at 03:38