0

I encountered an oscillating training loss curve problem. So I followed some tutorials and tried to solve it.

enter image description here

I did a learning rate test, and the LR ranges from 1E-7 to 0.1. The network I used is ResNet-50, and the optimizer is Adam.

Below is the LRTest result I got.

enter image description here

What I expected is a curve that first decreases slowly, then decreases steeply, and at last goes up. But I got this different result. I don't know what this trend of curve means. What should I do to select an appropriate learning rate?

Kitty
  • 29
  • 5
  • Thanks for your reply. I just wonder that if I use Cyclic Learning Rate, how should I set min_lr and max_lr? According to this LRTest curve plot, maybe the min_lr=1e-7 and max_lr=0.05? – Kitty Nov 23 '22 at 06:47

0 Answers0