How to change the learning rate in PyTorch (1.6)

Question

I am using PyTorch and I want to change the learning rate after some epochs.

However, the code that is provided on most documentations, which is:

  optimizer = torch.optim.Adam([
        dict(params=model.parameters(), lr=learning_rate),
    ])
   #This line specifically
   optimizer.params_group[0]['lr'] = learning_rate

does not work.

Actually PyCharm hints at it:

Unresolved attribute reference 'params_group' for class 'Adam'

As a result, the error thrown is:

AttributeError: 'Adam' object has no attribute 'params_group'

How should one manually change the learning rate in PyTorch (1.6)?

What a silly one. Been following some tutorial and I haven't even noticed. — Timbus Calin, Aug 21 '20 at 19:16

score 1 · Accepted Answer · edited Aug 22 '20 at 18:13

1

Param_groups is not the feasible solution devised by the pytorch and thus you should be implementing pytorch.optim.lr_scheduler. Read more about this at other stackoverflow answer here.

import torch.optim.lr_scheduler.StepLR #step learning rate
scheduler = StepLR(optimizer, step_size=5, gamma=0.1)

edited Aug 22 '20 at 18:13

Timbus Calin

13,809
5
41
59

answered Aug 21 '20 at 21:59

Mughees

832
7
16

score 0 · Answer 2 · answered Aug 21 '20 at 19:16

0

In the comment above, @Alizera Omidi provided the answer.

There was a typo in the params_group, which is in fact param_groups

answered Aug 21 '20 at 19:16

Timbus Calin

13,809
5
41
59

How to change the learning rate in PyTorch (1.6)

2 Answers2