Freezing conv layers in pre trained VGG16 model

Question

model_conv = torchvision.models.vgg16(pretrained=True)
for param in model_conv.parameters():
    param.requires_grad = False
model_conv.classifier.requires_grad_=True
model_conv.classifier[6].out_features=len(class_names)

model_conv = model_conv.to(device)

criterion = nn.CrossEntropyLoss()

optimizer_conv = optim.SGD(model_conv.classifier.parameters(), lr=0.001, momentum=0.9)

exp_lr_scheduler = lr_scheduler.StepLR(optimizer_conv, step_size=7, gamma=0.1)

Here in the VGG16 model, I want to train the classifier layer on my images and freeze the convolution layers. I am getting the same error.

RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

Be mindful of giving feedback to answers, this is the fourth time I am answering one of your questions without receiving any response whatsoever (see [here](https://stackoverflow.com/questions/69222705/use-of-dim-0-1-in-pytorch-and-nn-softmax), [here](https://stackoverflow.com/questions/69393281/freezing-conv-layers-in-pre-trained-vgg16-model), [here](https://stackoverflow.com/questions/69220221/use-of-torch-stack), and [there](https://stackoverflow.com/questions/68542593/runtimeerror-mat1-and-mat2-shapes-cannot-be-multiplied-28x28-and-784x64)...). — Ivan, Oct 03 '21 at 16:38
Sorry mate, I will now remind myself to respond to the answers. I generally forget to mark them as solutions.Will do from now on. — Dhruv Vashist, Oct 07 '21 at 14:19

score 0 · Answer 1 · answered Sep 30 '21 at 14:43

First of all requires_grad_ is an inplace function, not an attribute you can either do:

>>> model_conv.classifier.requires_grad_(True)

Or modify the requires_grad attribute directly (as you did in the for loop):

>>> model_conv.classifier.requires_grad = True

Second, you can't change the number of neurons in the layer by overwriting out_features. This attribute just contains the number of neurons and as no effect on the underlying content of the layer. You need to overwrite the layer with a newly initialized layer. For example:

>>> classifier = model_conv.classifier[6]
>>> model_conv.classifier[6] = nn.Linear(classifier.in_features, class_names)

A minimal example would be:

model_conv = torchvision.models.vgg16(pretrained=True).requires_grad_(False)
clf = model_conv.classifier[6]
clf.out_features=nn.Linear(clf.in_features, len(class_names))

optimizer_conv = optim.SGD(model_conv.classifier.parameters(), lr=0.001, momentum=0.9)
x = torch.rand(1,3,100,100, requires_grad=True)
model_conv(x).mean().backward()
optimizer_conv.step()

Freezing conv layers in pre trained VGG16 model

1 Answers1