Limiting the time that a function processes in an R for loop

Question

I want to apply a function ("foo" for this explanation) to tranform a vector of data into another value. This function takes the data as an input, and needs submit forms to webpages. Sometimes, this goes quickly, and other times, it can a long time. I would like to run the for loop (or equivalent apply function) in a way that skips over the items that take too long. I have tried to limit the time the loop runs before skipping to the next to 5 seconds using the following:

pb <- txtProgressBar(min = 1, max = 100, style = 3)
storage <- matrix(nrow = sample.length, ncol = 2)

for(i in 1:100){  
     s <- Sys.time()  
     storage[i,] <- try(foo(data.vec[i]), TRUE)  
     if (Sys.time() - s >5) {next}  
     # update progress bar  
     setTxtProgressBar(pb, i)  
}  
close(pb)

I think that I must not be understanding how to apply the 'next' condition in a for loop. have searched to find a clearer explanation, but not getting any luck here.

Your foo() does not return to Sys.time before it is finished. I you want to achieve the skipping, you have to add similar code in the foo function; or run foo in a separate thread, but that's not easy. — Dieter Menne, Dec 28 '11 at 07:02
The `evalWithTimeout` example is the way to go. Otherwise: Perhaps the actual function you're using to call web pages (or whatever) has a built-in timeout argument? Worth digging into the documentation if you haven't done so already. — Carl Witthoft, Dec 28 '11 at 16:18

score 14 · Accepted Answer · edited Mar 12 '19 at 13:39

14

withTimeout() from package R.utils, in concert with tryCatch(), might provide a cleaner solution.

For example:

require(R.utils)

for(i in 1:5) {
    tryCatch(
        expr = {
            withTimeout({Sys.sleep(i); cat(i, "\n")}, 
                         timeout = 3.1)
            }, 
        TimeoutException = function(ex) cat("Timeout. Skipping.\n")
    )
}

# 1 
# 2 
# 3 
# Timeout. Skipping.
# Timeout. Skipping.

In the artificial example above:

The first argument to withTimeout() contains the code to be evaluated within each loop.
The timeout argument to withTimeout() sets the time limit in seconds.
The TimeoutException argument to tryCatch() takes a function that is to be executed when an iteration of the loop is timed out.

edited Mar 12 '19 at 13:39

David

3
2

answered Dec 28 '11 at 07:22

Josh O'Brien

159,210
26
366
455

2

My loop is breaking. How to resume it? ` Error in checkError(res) : Undefined error in httr call. httr output: Operation was aborted by an application callback ` – Peter.k Aug 26 '18 at 09:34
Same issue here. Did you manage to solve it? – Fernando C. May 25 '22 at 11:12

score 0 · Answer 2 · answered Feb 27 '23 at 21:59

I am working on getting a loop to work in foreach with %dopar% so running an error message has caused me a headache, since this will break the loop.

I create the skip variable each time, and when timeout occurs, the loop goes to the next iteration. Hopefully avoiding any 'stop' or 'break' when used with partnering functions.

    for(i in 1:5) {
     res=NULL
     skip=FALSE
    tryCatch(
      {res= withTimeout({Sys.sleep(i); 
       cat(i, "\n")},  timeout = 3.1, 
       onTimeout='error')
        }, 
     TimeoutException = function(ex){skip <<- 
        TRUE})
    if(skip){next}
}

Limiting the time that a function processes in an R for loop

2 Answers2

Linked