0

I am using a complicated model which potentially would takes a long computing time. I want to terminate the fitting process if it reached a given MAX_TIMEOUT time.

I saw some threads about this issue and found a few solutions.

  1. using "threading" : I tried the following example but this did not work.
  2. using "multiprocessing" : While this approach works successfully if it ran on the main thread(inside the if _name_ == "main" block), it does not work under an interactive environment such as Jupyter Lab.
  3. using "signal" : seems not feasible because I am using a Windows machine(Some methods only work on Unix systems)
  4. using "timeout-decorator" : Due to some business restriction, I cannot use this library. Also this package uses "signal", hence it seems not work on Windows
  5. using "wrapt_timeout_decorator" : This also not work if I ran script on Jupyter Lab. I suppose this is because of the same reason as "multiprocessing" which this library relies on.

Is there a way to solve this problem?

I am using Jupyter Lab on a Windows machine.

ref

Code example

import numpy as np
import statsmodels.api as sm

y = np.random.rand(5000)

# Complicated model
mod = sm.tsa.UnobservedComponents(
    y, 
    level = True, stochastic_level = True, 
    freq_seasonal = [{'period' : 200, 'harmonics' : 50}], stochastic_freq_seasonal = [True]
)


# Fitting the model(this will take more than 1 min)
fit = mod.fit()

def fit_with_timeout(mod, MAX_TIMEOUT) : 
    """
    Call mod.fit(). 
    If the computation succeeded within MAX_TIMEOUT, then return mod.fit(), otherwise this function should return None
    How I can solve this?
    """

The following is my threading approach that does not work. With this approach,

  1. the target function will still be alive even after 'raise' triggered by terminate_alive_thread(t).
  2. 'raise' will not be caught by the try-except block, which I want to do some work in the except block if the target function get timeout.
import threading
import time

# Some time-consuming function
def f(x) : 
    print(x, 'start')
    time.sleep(x)
    with open(str(x) + '.txt', 'w') as file : 
        file.write(str(x))
    return x

def terminate_alive_thread(t) : 
    if t.is_alive() : 
        raise 

# target function that will take 5 sec to finish
t_target = threading.Thread(target = f, args = (5, ))

# Set timer to terminate the target function 
timeout = 1
timer = threading.Timer(timeout, terminate_alive_thread, args = (t_target, ))

timer.start()
try : 
    t_target.start()
except : 
    print('Do some work in this block')
Shuma
  • 1
  • 1

0 Answers0