0

I am trying to print the mean_test_score after having fit Randon Forest models.

The traceback error is below and I understand that the issue with this is that .mean_test_score cannot be used on score because it's a string.

I have seen a similar question here and have tried to follow the debugging process myself without success.

I also understand mean_validation_score was deprecated hence why I have used mean_test_score as explained here.

I have tried debugging this myself but I am not sure of what to try next in order to get the printed mean_test_scores from the fitted models.

Could someone provide some guidance or point me in the direction of some useful resources to resolve this?

Please note I am using Python 3.8.8 in Jupyter Notebook

EDIT: Example code from scitkit learn, although some elements have been deprecated https://scikit-learn.org/0.15/auto_examples/randomized_search.html

def report(grid_scores, n_top=3):
    top_scores = sorted(grid_scores, key=itemgetter(1), reverse=True)[:n_top]
    for i, score in enumerate(top_scores):
        print("Model with rank: {0}".format(i + 1))
        print("Mean Test score: {0:.3f} (std: {1:.3f})".format(
              score.mean_test_score,
              np.std(score.cv_validation_scores)))
        print("Parameters: {0}".format(score.parameters))
        print("")

clf = RandomForestClassifier()

n_iter_search = 20
random_search = RandomizedSearchCV(clf, param_distributions=param_dist,
                                   n_iter=n_iter_search)
start = time()
random_search.fit(features, target)

print("RandomizedSearchCV took %.2f seconds for %d candidates"
      " parameter settings." % ((time() - start), n_iter_search))

RandomizedSearchCV took 4432.59 seconds for 20 candidates parameter settings. .cv_results_

report(random_search.cv_results_)

Traceback error:

Model with rank: 1
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-107-14a2b2b295f5> in <module>
----> 1 report(random_search.cv_results_)

<ipython-input-103-4d4244087b69> in report(grid_scores, n_top)
      4         print("Model with rank: {0}".format(i + 1))
      5         print("Mean Test score: {0:.3f} (std: {1:.3f})".format(
----> 6               score.mean_test_score,
      7               np.std(score.cv_validation_scores)))
      8         print("Parameters: {0}".format(score.parameters))

AttributeError: 'str' object has no attribute 'mean_test_score'
WPB
  • 13
  • 4
  • `top_scores` is a list of strings. I think the attributes you are looking for are in `grid_scores` – balderman Aug 26 '21 at 12:26
  • @balderman I have tried iterating through grid_scores instead of top_scores. Unfortunately the same error is returned – WPB Aug 26 '21 at 13:02
  • do `print(type(top_scores[0]))` and `print(type(grid_scores[0]))` - what do you get? – balderman Aug 26 '21 at 13:06
  • A bit peculiar but these errors: NameError: name 'top_scores' is not defined and NameError: name 'grid_scores' is not defined – WPB Aug 26 '21 at 13:21
  • Do it in report function – balderman Aug 26 '21 at 13:27
  • KeyError Traceback (most recent call last) in ----> 1 report(random_search.cv_results_) in report(grid_scores, n_top) 2 top_scores = sorted(grid_scores, key=itemgetter(1), reverse=True)[:n_top] 3 print(type(top_scores[0])) ----> 4 print(type(grid_scores[0])) 5 6 for i, score in enumerate(top_scores): KeyError: 0 Does this mean that grid_scores contains nothing? – WPB Aug 26 '21 at 13:46
  • "I understand that the issue with this is that .mean_test_score cannot be used on score because it's a string." Okay - did you expect otherwise? Why? "Does this mean that grid_scores contains nothing? " It means that `grid_scores` **is a dict and not a list**, and that it doesn't have a value corresponding to the `0` key. – Karl Knechtel Aug 26 '21 at 21:59

0 Answers0