In the Pytorch documentation https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html#sphx-glr-beginner-blitz-autograd-tutorial-py In the image, I am unable to understand what y.backward(v) means and why do we need to define another tensor v to do the backward operation and also how we got the results of x.grad Thanks in advance
y.backward() computes dy/dz where z are all the leaf nodes in the computation graph. And it stores dy/dz in z.grad.
For example: In the above case, leaf nodes are x.
y.backward() works when y is a scalar which is the case for most of the deep-learning. When y is a vector you have to pass another vector (v in the above case). You can see this as computing d(v^Ty)/dx.
To answer how we got x.grad note that you raise x by the power of 2 unless norm exceeds 1000, so x.grad will be v*k*x**(k-1) where k is 2**i and i is the number of times the loop was executed.
To have a less complicated example, consider this:
x = torch.randn(3,requires_grad=True)
print(x)
Out: tensor([-0.0952, -0.4544, -0.7430], requires_grad=True)
y = x**2
v = torch.tensor([1.0,0.1,0.01])
y.backward(v)
print(x.grad)
Out[15]: tensor([-0.1903, -0.0909, -0.0149])
print(2*v*x)
Out: tensor([-0.1903, -0.0909, -0.0149], grad_fn=<MulBackward0>)
