Add regression layer to MatConvNet

Question

I have designed a smile detection system. this system is based on deep learning and has been implemented by MatConvnet. The last layer is the output of the system and has 10 output according to the amount of the person's smile. I want to convert these ten output with a numeric output in the range of 1 to 10 with a regression layer. How can I do this in MatConvNet. Thanks

Answer 1

For regression layer, you will have to replace the final layer (usually softmax or softmaxloss) of the CNN with your own layer for regression.

In order to define your own layer in MatConvNet, you need to have the following:

1. Loss function
2. Gradient of this loss function

A popular choice for a regression loss function is the squared error loss. You should take a look at how softmaxloss is implemented in MatConvNet and how it is called in examples/cnn_train.m. More details can be found in the MatConvNet docs and in a short tutorial on backpropagation on squared loss here.

Suggestion: This does not answer your question. However, given your problem, it is easier to convert your 10 output classification to a score rather than train the whole CNN for regression.

Assign numbers 1 to 10 to your 10 classes of smiles. The CNN would give a posterior probability P(1...10) over all these classes for an input image. Simply take the weighted sum over the class labels ( 1*P(1) + 2*P(2) + ....) to get a real-valued number for the amount of smile.