2

I have a code block, to perform 3 times retry of the code execution in case of a specific error. In below example if HTTP 503 error occurred during the data download from ADLS container, I want the same operation to be executed maximum of 3 times retry.

require(AzureStor)
require(stringr)
recheck <- 0
while (recheck < 3){
  recheck <- recheck + 1
  tryCatch({
    storage_download(container, file, filename, overwrite=TRUE)
    recheck <- 4  
  }, error = function(e){
    if ( sum(str_detect(e, '503')*1) > 0 ){
      print(e)
      print(paste0('An infra-level failure occured. Retry sequence number is : ', recheck))
    } else{
      recheck <<- 4
      print(e)
    }
  }
  )
}

This code works fine for me, but similar to storage_download in the above example, I have other ADLS operations like delete_blob, upload_blob, storage_upload, list_storage_files at multiple instances in the code, I have to write above mentioned code for each of these functions. I want to make the above code as a function which can be called during each of these ADLS operations. Any thoughts or suggestions would help me greatly.

msr_003
  • 1,205
  • 2
  • 10
  • 25
  • 1
    [Regarding your use of `require`](https://stackoverflow.com/a/51263513/1968) – Konrad Rudolph Jan 17 '23 at 09:24
  • And what is the logic of `sum(str_detect(e, '503')*1) > 0`? You can just write `str_detect(e, '503')`. – Konrad Rudolph Jan 17 '23 at 09:36
  • I have some error code words in a vector. I want the if condition to work only on presence of any of the vector item. To simply the question I have just mentioned HTTP 503 error here in the example. – msr_003 Jan 17 '23 at 09:49
  • Hmm the `e` argument in the `tryCatch` `error` callback should only ever be a single error object. In fact, the code should use `conditionMessage(e)` instead of just `e`, my bad. — But even if `e` is a *vector*, your code is needlessly convoluted. Instead, use: `any(str_detect(e, '503'))`. – Konrad Rudolph Jan 17 '23 at 09:51
  • 2
    @KonradRudolph's answer below will solve this particular issue, but for a general approach that you could use in future projects it's worth having a read of this: https://adv-r.hadley.nz/function-factories.html – Paul Stafford Allen Jan 17 '23 at 09:51

1 Answers1

2

The following should do the trick:

with_retries_on_failure = function (expr, retries = 3L) {
    expr = substitute(expr)

    for (try in seq_len(retries)) {
        tryCatch(
            return(eval.parent(expr)),
            error = \(e) {
                if (str_detect(conditionMessage(e), '503')) stop(e)
                message('An infra-level failure occurred. Retry sequence number is: ', try)
            }
        )
    }
}

Used as follows:

with_retries_on_failure(storage_download(container, file, filename, overwrite=TRUE))

Note the return() call, which immediately returns from the surrounding function without the need to update the loop variable. Likewise, in the case of a failure we also don’t have to update the loop variable since we are using a for loop, and we use stop() to break out of the loop for any error that is not a 503 HTTP response.

Konrad Rudolph
  • 530,221
  • 131
  • 937
  • 1,214
  • Thanks for the quick help. I believe I'm missing some dependencies in my R-studio. I'm getting code errors with `{` and `:` – msr_003 Jan 17 '23 at 09:52
  • @Konard Rudolph, That's a beautiful code block you have given. I have R vesrion 4.1.2, unfortunately I'm still getting same errors. I ll ask my IT team to upgrade to latest version of R. – msr_003 Jan 17 '23 at 11:10
  • @msr_003 Ugh, apologies, I just noticed a typo in the code. `:` should be `in`, obviously. – Konrad Rudolph Jan 17 '23 at 13:09
  • This is so perfect, just had to remove `!` in if condition to meet my need. Thank you!! – msr_003 Jan 18 '23 at 05:43