2

I am using Caffe and also NVIDIA DIGITS. I want to use AlexNet pretrained on ImageNet and wanna fine tune it on my medical data. I have nearly 1000 images and using 80% for training, I generated 40,000 images by data augmentation (using cropping and rotation). However I face a severe overfitting. I tried to overcome this by adding multiple dropout layers. and the result change from :

results 1

to:

results 2

but my accuracy does not improve.

my network specifications:

AlexNet pre-trained on ImageNet

base learning rate: 0.001

learning rate multiplier: 0.1 for convolution layers and 1 for fully connected layers and xavier weight initialisation.

dropout: 0.5

Now I want to add L2 regularization. I did not find such layer in Caffe and I should maybe make it myself.

first question: Do you have any solution for my problem? ( I have tried other ways like changing stepsize, changing learning rate from 1 to 10^(-5) and I found 0.001 is better, weigh decay changes, adding various dropout layer (which helped as you see))

second question: can you please help me how I can implement L2 regularization??

Shai
  • 111,146
  • 38
  • 238
  • 371
azzz
  • 91
  • 7

1 Answers1

1

You have L2 regularization by default in caffe.
See this thread for more information.

Community
  • 1
  • 1
Shai
  • 111,146
  • 38
  • 238
  • 371