Sklearn Predict_Proba: ValueError: operands could not be broadcast together with shapes (44,57) (4,)

Question

I am trying to computer the log_loss of a machine learning model through recursive feature elimination and a value error occurs when I try to run sklearns predict_proba function

I have currently tried to remove the code that refits my dataset through RFE. When I do that, so that no shape altering occurs, the same error still happens. Test data starts at a shape of (44,2048) Test Labels starts at (44,) After being refit it changes to: Test Data shape becomes (44,57) Test Labels is not effected

I'm really at a loss here of what to try next any suggestions would be helpful. I don't understand where the (4,) comes from in terms of it trying to shape my data into it

def get_log_loss(self, test_data, test_labels, rfe):
    rfe.fit(test_data, test_labels)
    test_data = rfe.transform(test_data)
    test_probs = self._model.predict_proba(test_data)
    score = log_loss(test_labels, test_probs)
    return score

Error:

Traceback (most recent call last):
      File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.1.2\helpers\pydev\pydevd.py", line 2060, in <module>
        main()
      File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.1.2\helpers\pydev\pydevd.py", line 2054, in main
        globals = debugger.run(setup['file'], None, None, is_module)
      File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.1.2\helpers\pydev\pydevd.py", line 1405, in run
        return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
      File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.1.2\helpers\pydev\pydevd.py", line 1412, in _exec
        pydev_imports.execfile(file, globals, locals)  # execute the script
      File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.1.2\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
        exec(compile(contents+"\n", file, 'exec'), glob, loc)
      File "C:/Users/myname/PycharmProjects/MQP/mainv1.py", line 376, in <module>
        test_model(test_batch, test_labels, j, l)
      File "C:/Users/myname/PycharmProjects/MQP/mainv1.py", line 190, in test_model
        logloss = model_obj.get_log_loss(a_test_batch, a_test_labels, rfe)
      File "C:\Users\myname\PycharmProjects\MQP\trainData.py", line 59, in get_log_loss
        test_probs = self._model.predict_proba(test_data)
      File "C:\Users\myname\AppData\Local\Continuum\anaconda3\lib\site-packages\sklearn\naive_bayes.py", line 104, in predict_proba
        return np.exp(self.predict_log_proba(X))
      File "C:\Users\myname\AppData\Local\Continuum\anaconda3\lib\site-packages\sklearn\naive_bayes.py", line 84, in predict_log_proba
        jll = self._joint_log_likelihood(X)
      File "C:\Users\myname\AppData\Local\Continuum\anaconda3\lib\site-packages\sklearn\naive_bayes.py", line 437, in _joint_log_likelihood
        n_ij -= 0.5 * np.sum(((X - self.theta_[i, :]) ** 2) /
    ValueError: operands could not be broadcast together with shapes (44,57) (4,)

Refer the same question related to it - [here](https://stackoverflow.com/questions/24560298/python-numpy-valueerror-operands-could-not-be-broadcast-together-with-shapes) — Rohit Gurjar, Aug 02 '19 at 19:03
print and tell me the shape of `test_data `, `test_labels `, and `test_data ` after transform — seralouk, Aug 02 '19 at 20:06

Sklearn Predict_Proba: ValueError: operands could not be broadcast together with shapes (44,57) (4,)

0 Answers0