How does RandomSearchCV handle models/memory?

Question

I was wondering how sklearn.model_selection.RandomizedSearchCV (and for GridSearchCV in an extent) was handling models, info and memory. In fact, despite some researches, I couldn't find any resources explaining (either with n_jobs =-1 or >1) what was stored every stacking step.

Even though I'm pretty sure that every model is stored because you can retrieve the model with the best parameters, but does it keep every one of them? Or does it keep at every step the best one between the last constructed one with the stored one?

Thanks in advance for your responses :)

Answer 1

They does not keep any intermediate model. Only the hyper-parameters and output metrics associated with each model are stored. The model with the best parameters that you can retrieve is trained in the very end on whole data (not cross-validated, because thats already been done).

You can look at my other answers which describe the GridSearchCV in more detail:

• Model help using Scikit-learn when using GridSearch
• scikit-learn GridSearchCV with multiple repetitions

RandomizedSearchCV does the same except for the first part where the candidate parameter combinations are found out.