Compute log loss for logistic regression from scratch

Question

compute log-loss

def logloss(y_true,y_pred):
    '''In this function, we will compute log loss '''
    log_loss = (-((y_true * np.log10(y_pred)) + (1-y_true) * np.log10(1-y_pred)).mean())
    return log_loss

Computing logistic regression

def train(X_train,y_train,X_test,y_test,epochs,alpha,eta0):
 w,b = initialize_weights(X_train[0])
    train_loss = []
    test_loss = []
    for e in range(epochs):
        for x,y in zip(X_train,y_train):
            dw = gradient_dw(x,w,y,b,alpha,N)
            db = gradient_db(x,y,w,b)
            w = w + (eta0 * dw)
            b = b + (eta0 * db)
        train_pred = []
        for i in X_train:
            y_pred = sigmoid(np.dot(w.T, i) + b)
            train_pred.append(y_pred)
        train_loss.append(logloss(y_train, train_pred))
        
        test_pred = []
        for j in X_test:
            y_pred_test = sigmoid(np.dot(w.T, j) + b)
            test_pred.append(y_pred_test)
        test_loss.append(logloss(y_test, test_pred))
    return w,b

alpha=0.0001
eta0=0.0001
epochs=50
N = len(X_train)
w,b = train(X_train,y_train,X_test,y_test,epochs,alpha,eta0)

Error that I am getting

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-112-9a34879eb072> in <module>
      3 epochs=50
      4 N = len(X_train)
----> 5 w,b = train(X_train,y_train,X_test,y_test,epochs,alpha,eta0)

<ipython-input-110-db0e3d88382d> in train(X_train, y_train, X_test, y_test, epochs, alpha, eta0)
     30             y_pred = sigmoid(np.dot(w.T, i) + b)
     31             train_pred.append(y_pred)
---> 32         train_loss.append(logloss(y_train, train_pred))
     33 
     34         test_pred = []

<ipython-input-108-f272288a384c> in logloss(y_true, y_pred)
      1 def logloss(y_true,y_pred):
      2     '''In this function, we will compute log loss '''
----> 3     log_loss = (-((y_true * np.log10(y_pred)) + (1-y_true) * np.log10(1-y_pred)).mean())
      4     return log_loss

TypeError: unsupported operand type(s) for -: 'int' and 'list'

I have mentioned the full code just the codes for which I am getting error.I am confused whether to make changes to logloss or make changes to logistic regression code i.e def train(). How to rectify this error?

try printing `y_true` and `y_pred` in `logloss()` function. And show the result — Prakash Dahal, Jun 08 '21 at 11:41
Irrelevant to your issue, but we *never* program the actual mathematical formula as-is for log loss, as it will certainly result in logarithms of zeros; we always [clip the inputs](https://stackoverflow.com/questions/65759770/intuition-behind-categorical-cross-entropy/65760180#65760180) first. — desertnaut, Jun 08 '21 at 14:17
its like an assignment for me for I am stuck in constructing this algorithm. @desertnaut — maddy, Jun 15 '21 at 08:49
assignment or not is irrelevant; and part of such assignments is exactly how you should "translate" the formula in the appropriate way. Did you check the suggested link? — desertnaut, Jun 15 '21 at 08:55
And are you aware that log loss and cross-entropy are actually [synonyms](https://stackoverflow.com/questions/50913508/what-is-the-difference-between-cross-entropy-and-log-loss-error)? — desertnaut, Jun 15 '21 at 09:02
No, I was not aware of that because I haven't studied about the cross-entropy yet. And also in the assignment I can't use cross-entropy now as it part of SGD classifier for log loss for logistic regression. These assignments are designed in a way for particular concepts. @desertnaut — maddy, Jun 15 '21 at 09:58
I am just trying to say that the functions I have shown [here](https://stackoverflow.com/questions/65759770/intuition-behind-categorical-cross-entropy/65760180#65760180) do you job exactly, simply & beautifully. Not sure what you mean you can't use cross-entropy - you are doing exactly that, because it is the same thing with log loss. — desertnaut, Jun 15 '21 at 10:21

score 1 · Answer 1 · answered Jun 08 '21 at 17:16

1

Your train_pred is python list. When you use it in logloss function, you calculate (1-train_pred), which is integer minus python list. Therefore you get type error

TypeError: unsupported operand type(s) for -: 'int' and 'list'

answered Jun 08 '21 at 17:16

lanenok

2,699
17
24

1

Thanks for the reply. So, could you help me out and tell me what correction have to be done. @lanenok – maddy Jun 15 '21 at 08:48

Compute log loss for logistic regression from scratch

1 Answers1