How to use pytorch to perform gradient descent, and print out the true minimum?

Question

I'm trying to use a gradient method to find a minimizer of a function f, where x(k+1)=x(k)-α▽f(x), and α=0.1. The function f is f(x) = 2*(x^3+1)^(-1/2), x(0)=1 Here is the pytorch sample

import torch
import torch.nn as nn
from torch import optim

alpha = 0.1

class MyModel(nn.Module):
     def __init__(self):
          super().__init__()
          self.x = 1

def forward(self, x):
     return 2*pow((pow(x,3)+1),(-1/2))

model = MyModel()
optimizer = optim.SGD(model.parameters(),lr=alpha)
terminationCond = False

while not terminationCond:
     f = model.forward(x)
     f.backward()
     optimizer.step()
     if x.grad < 0.001:
           terminationCond = True
     optimizer.zero_grad()

But I cannot output the correct value of x, how to modify my code in order to find a minizer of a function f?

There's a typo in `modef.forward(x)` I suppose? And the `forward` function would also need to be properly indented?! Also, how do you prevent `x` from becoming negative during the optimization, that wouldn't be allowed, right? I guess you could try different learning rates if you haven't yet. — Jan, Jun 13 '22 at 01:58
Check this [post](https://stackoverflow.com/questions/55604691/how-to-use-autograd-find-min-max-point) — The Exile, Jun 13 '22 at 11:47

score 1 · Answer 1 · answered Jun 13 '22 at 13:40

There are a few things that need to be considered ...

For x to be a parameter (model.parameter()) x should be nn.Parameter(torch.as_tensor([1.]))
You are passing what (and why) x to the forward method?
The function f(x) = 2*(x^3+1)^(-1/2) is inversely proportional to x. When x goes up in value, the function goes down. The SGD will minimize the value of the function, hence maximize x.

f(0) = 2.0
f(1) = 1.41421
f(2) = 0.66666

Here is a working example of minimizing f(x) ie maximizing x.

import torch
import torch.nn as nn
from torch import optim

alpha = 0.1

class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.x = nn.Parameter(torch.as_tensor([1.0]))
    def forward(self):
        return 2*pow((pow(self.x, 3)+1), (-1/2))

model = MyModel()
optimizer = optim.SGD(model.parameters(), lr=alpha)
terminationCond = False
print(model.x)

f = model()
f.backward()
optimizer.step()
optimizer.zero_grad()
print(model.x)

How to use pytorch to perform gradient descent, and print out the true minimum?

1 Answers1