Mlflow it is possible to log confusion matrix every step?

Question

It is possible to log with mlflow the confusion matrix every step like a simple metrics? If it is possible it have a visualization like this?

You don't add much information about the libraries and functions you're using. Anyway, here is a [worked out example](https://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html#sphx-glr-auto-examples-model-selection-plot-confusion-matrix-py) using scikit-learn. — JohanC, Mar 11 '20 at 10:05
[This answer](https://stackoverflow.com/a/51176855/12046409) might be helpful. — JohanC, Mar 11 '20 at 10:12
My question is if it is possible to log it as a metric or something else in mlflow, or I brutally create my file and log it as artifact... — Nicola Landro, Mar 12 '20 at 10:35

score 3 · Answer 1 · answered Mar 17 '20 at 23:28

For every possible run, you could get the individual confusion matric values

# get confusion matrix values
conf_matrix = confusion_matrix(y_test,y_pred)
true_positive = conf_matrix[0][0]
true_negative = conf_matrix[1][1]
false_positive = conf_matrix[0][1]
false_negative = conf_matrix[1][0]

mlflow.log_metric("true_positive", true_positive)
mlflow.log_metric("true_negative", true_negative)
mlflow.log_metric("false_positive", false_positive)
mlflow.log_metric("false_negative", false_negative)

And then log_artifact(<your_plot>, "confusion_matrix")

Not a good solution for a multi-class problem. – Amin Tabatabaei Aug 09 '22 at 09:50 — Amin Tabatabaei, Aug 09 '22 at 09:50

score 0 · Answer 2 · answered Feb 13 '23 at 23:07

Since you already have the confusion matrix as visualization you can log it with the mlflow.log_artifact() call as file.

The official documenation also has an example even though it is with a txt file it should be no problem to serialize the visualization somehow.

Mlflow it is possible to log confusion matrix every step?

2 Answers2