How can I assign a time limit to randomsearchCV (running forever)?

Question

Is it possible to assign a time limit to randomsearch cv? I know that you can tweak number of iterations, but I specifically want to control maximum searching time besides the number of iterations.

The reason behind this is that am using a search grid of different preprocessing steps and for every preprocessing combination, I call a randomsearchCV with cv=4 and iterations=30. This is no problem for most combinations (training time really short <20s), but for some combinations it seems to run forever and I don't know why.

I searched for hours what the problem behind the infinite training time was, but I have given up on this so I just want to skip cases where the training time exceeds 10 minutes.

I can show code if necessary, but I think it wouldn't help here.

score 0 · Accepted Answer · answered Apr 02 '20 at 08:59

0

I am not sure which module you are using and whether there is an implemented function to do what you want, but you could "manually" check on the time passed in your code:

import time

def somefunc(time_started):
    a = true
    time_limit = 5000
    while a: 
         # do something / run an iteration
         if round(time.monotonic() * 1000) - time_started > time_limit: 
              # check how much time/ms has elapsed since you called the function
              a = false # will break on next "while-iteration" 

t0= round(time.monotonic() * 1000)
func(t0)

Obviously, you can change time_limit to any other amount you need for your purpose.

answered Apr 02 '20 at 08:59

Cribber

2,513
2
21
60

Good suggestion. However, I want to interrupt the sklearn.randomizedsearchcv function while it is still running if the running time exceeds 10 minutes. This means that that function first has to end, before I can check the current running time. The idea that you propose works well if the task has completed, but that's not the case here. – Tim Smit Apr 02 '20 at 09:11
ah, got it - checkout this post, it works with the module multiprocessing https://stackoverflow.com/questions/14920384/stop-code-after-time-period. I am not sure though if "killing" your process is the right thing or if you need it to stop & do some "cleanup" after the current iteration? – Cribber Apr 02 '20 at 09:16

How can I assign a time limit to randomsearchCV (running forever)?

1 Answers1