The MLFlow Tracking is great for monitoring experiments, but I wonder if there is a solution on MLFlow or another open-source platform that can be integrated to monitor data and model drift.
There is a post from Databricks showing how to achieve that with Delta Lake, however, as you can deploy and serve models with MLFlow, it looks to me that it would be easy to monitor the predictions made by the model, the same way we monitor the experiments run.
My team has recently added integration between MLflow and our open source data monitoring library called whylogs. This lets you log statistical profiles of the data passing through the model and/or the output of the model. You can then collect these profiles from MLflow run artifacts and analyze them for drift.
We have a notebook that walks you through the integration process and a blog post to go along with it. Lmk if you have any questions or additional feature requests!
