Convert this loss function equation into python code

Question

Please check this equation of this link and convert it into a python loss function for a simple keras model.

EQUATION PICTURE OR IMAGE LINK FOR CONVERTING IT TO PYTHON'S KERAS REQUIRED LOSS EQUATION

where the max part or the curve selected part of the equation in the picture is the hinge loss, yi represents the label of each example, φ(x) denotes feature representation, b is a bias, k is the total number of training examples and w is the classifier to be learned.

For easy check, the sample equation is -

min(w) [
1/k(sum of i to k)
max(0, 1 - y_i(w.φ(x) - b))
]
+
1/2||w||^ 2 
.

Actually I can find the max part or the curved section of the equation in the picture but I can not find the 1/2 * ||w||^ 2 part.

You check this link too for help -

similar link

Here I have attached some sample code to clear the concept of my issue:

print("Create Model")
model = Sequential()
model.add(Dense(512,     
input_dim=4096, init='glorot_normal',W_regularizer=l2(0.001),activation='relu'))
model.add(Dropout(0.6))
model.add(Dense(32, init='glorot_normal',W_regularizer=l2(0.001)))
model.add(Dropout(0.6))
model.add(Dense(1, init='glorot_normal',W_regularizer=l2(0.001),activation='sigmoid'))

adagrad=Adagrad(lr=0.01, epsilon=1e-08)     
model.compile(loss= required_loss_function, optimizer=adagrad)

def required_loss_function(y_true, y_pred): 
      IN THIS LOSS FUNCTION, 
      CONVERT THE EQUATION IN THE 
      PICTURE INTO PYTHON CODE.

AS A MENTION, THE THING YOU HAVE TO FIND IS THE- 1/2 * ||w|| ^ 2 . As I can find the python code of the remaining or other part of the equation in the linked picture. The hinge loss part can be easily calculated using this equation -

import keras

keras.losses.hinge(y_true, y_pred)

If you require further help, please comment for details.

Answer 1

Your screenshot shows the whole objective function, but only the sum(max(...)) term is called the loss term. Therefore, only that term needs to get implemented in required_loss_function. Indeed, you can probably use the pre-baked hinge loss from the keras library rather than writing it yourself—unless you're supposed to write it yourself as part of the exercise, of course.

The other term, the 0.5*||w||^2 term, is a regularization term. Specifically, it's an L2 regularization term. keras has a completely separate way of dealing with regularization, which you can read about at https://keras.io/regularizers/ . Basically it amounts to creating an l2 instance keras.regularizers.l2(lambdaParameter) and attaching it to your model with the .add() method (your screenshotted equation doesn't have a parameter that scales the regularization term–so, if that's literally what you're supposed to implement, it means your lambdaParameter would be 1.0).

But the listing you supply already seems to be applying l2 regularizers like this, multiple times in different contexts (I'm not familiar with keras at all so I don't really know what's going on–I guess it's a more sophisticated model than the one represented in your screenshot).

Either way, the answer to your question is that the regularization term is handled separately and does not belong in the loss function (the signature of the loss function gives us that hint too: there's no w argument passed into it—only y_true and y_pred).