1

In PyTorch, there are two ways of calculating second gradients. The first method is to use torch.autograd.grad function, and the other is to use backward function. I use the following examples to illustrate it:

Method 1:

x=torch.tensor([3.0], requires_grad=True)
y = torch.pow(x, 2)
grad_1 = torch.autograd.grad(y, x, create_graph=True)
print(grad_1[0].item())
grad_2 = torch.autograd.grad(grad_1[0], x)
print(grad_2)

The result makes sense for me, and the second gradient of the function is 2.

Method 2:

x=torch.tensor([3.0], requires_grad=True)
y = torch.pow(x, 2) # y=x**2
y.backward(retain_graph=True)
print(x.grad)

y.backward()
print(x.grad)

When calculating the first gradient, I use create_graph=True to make sure that we can use back prorogation method to calculate the second gradient. However, the result is is 12, which is wrong. I was wondering what's wrong with the second method?

feelfree
  • 11,175
  • 20
  • 96
  • 167
  • Does this answer your question? [PyTorch most efficient Jacobian/Hessian calculation](https://stackoverflow.com/questions/56480578/pytorch-most-efficient-jacobian-hessian-calculation) – iacob May 17 '21 at 22:10

1 Answers1

0

Use the grad method from torch.autograd to differentiate your function. So the steps would be:

>>> import torch
>>> from torch.autograd import grad

>>> x = torch.tensor([3.0], requires_grad=True)
>>> y = torch.pow(x,2)

>>> z = grad(y, x, create_graph=True)
>>> print(grad(z, x, create_graph=True))
>>> (tensor([2.], grad_fn=<MulBackward0>),)

Similarly, you can loop through to make the nth derivative.

Satya Prakash Dash
  • 807
  • 8
  • 16