How to combine future_promise with memoise in R plumber

Question

I am making an API with R package plumber. Some endpoints in this API would benefit from memoization, so I also use the memoise package. Combining both is pretty straight-forward, as demonstrated in this blog post. I'll build up on this post's example for my reprex.

# test.R
library(memoise)

# Define function ---------------------------------------------------------
add <- function(a, b) {
    a + b
}

# Memoisize function ------------------------------------------------------
add_mem <- memoise(add)

# Define API --------------------------------------------------------------
#* Return the sum of two numbers
#* @param a The first number to add
#* @param b The second number to add
#* @get /add
function(a, b) {
    add(as.numeric(a), as.numeric(b))
}

#* Return the sum of two numbers, use memoise cached results when applicable
#* @param a The first number to add
#* @param b The second number to add
#* @get /addWithMemoise
function(a, b) {
    add_mem(as.numeric(a), as.numeric(b))
}

# R console 1
plumber::plumb(file="test.R")$run(port=50000)
#Running plumber API at http://127.0.0.1:50000
#Running swagger Docs at http://127.0.0.1:50000/__docs__/

# R console 2
httr::GET("http://localhost:50000/add?a=1&b=1") %>% content()
#[[1]]
#[1] 2
httr::GET("http://localhost:50000/addWithMemoise?a=1&b=1") %>% content()
#[[1]]
#[1] 2

So now, I'd like to give my API the ability to handle concurrent calls. For this, I'll use the promises package, as demonstrated in this blog post. Combining plumber with promises is pretty straight-forward too:

# Added in test.R
library(promises)
#* Return the sum of two numbers, use promises
#* @param a The first number to add
#* @param b The second number to add
#* @get /addWithPromise
function(a, b) {
    future_promise({
        add(as.numeric(a), as.numeric(b))
    })
}

I restart the API, and then I can call this new endpoint:

# R console 2
httr::GET("http://localhost:50000/addWithPromise?a=1&b=1") %>% content()
#[[1]]
#[1] 2

Now I want to add an endpoint that mixes everything (plumber, memoise and promises).

# Added in test.R

#* Return the sum of two numbers, use promises and memoise cached results when applicable
#* @param a The first number to add
#* @param b The second number to add
#* @get /addWithPromiseMemoise
function(a, b) {
    future_promise({
        add_mem(as.numeric(a), as.numeric(b))
    })
}

I restart the API, then call it:

# R console 2
httr::GET("http://localhost:50000/addWithPromiseMemoise?a=1&b=1") %>% content()
#$error
#[1] "500 - Internal server error"
#
#$message
#[1] "Error in add_mem(as.numeric(a), as.numeric(b)): attempt to apply non-function\n"

# R console 1
#Unhandled promise error: attempt to apply non-function
#<simpleError in add_mem(as.numeric(a), as.numeric(b)): attempt to apply non-function>

I tried to mingle with future_promise's envir argument, with no success. I also tried this:

# Added in test.R

#* Return the sum of two numbers, use promises and memoise cached results when applicable
#* @param a The first number to add
#* @param b The second number to add
#* @get /addWithPromiseMemoiseTest
function(a, b) {
    print(names(environment(add_mem)))
    e <- environment(add_mem)
    future_promise({
        Sys.sleep(10)
        print(add_mem)
        print(class(add_mem))
        print(names(environment(add_mem)))
        environment(add_mem) <- e
        print(names(environment(add_mem)))
        add_mem(as.numeric(a), as.numeric(b))
    })
}

I restart the API, then call it:

# R console 2
httr::GET("http://localhost:50000/addWithPromiseMemoise?a=1&b=1") %>% content()
#[[1]]
#[1] 2

# R console 1
#[1] "_omit_args"    "_hash"         "_additional"   "_default_args" "_f_hash"       "_cache"        "_f"           
#Memoised Function:
#NULL
#[1] "memoised" "function"
# [1] "...future.startTime"    "...future.oldOptions"   "a"                      "b"                      "setdiff"                "...future.rng"          "e"                      "...future.frame"       
# [9] "...future.conditions"   "...future.mc.cores.old" "...future.stdout"       "add_mem"               
#[1] "_omit_args"    "_hash"         "_additional"   "_default_args" "_f_hash"       "_cache"        "_f"

It returns the right result, and the future_promise part works as expected (that I can see after adding a 10s Sys.sleep within the endpoint's future_promise call, then calling another future_promise endpoint from another R session), but the memoise part doesn't seem to work (that I can see after adding a 10s Sys.sleep within the add function, calling the last endpoint twice doesn't make it faster).

EDIT: this solution seems to work if I combine it with filesystem memoization:

# Changed in test.R

# Memoisize function ------------------------------------------------------
cache_fs <- cache_filesystem(".")
add_mem <- memoise(add, cache=cache_fs)

But it really doesn't look like a clean solution.

So as this is a long post, I'll state my question clearly in case I lost you:
Is there a clean way to combine memoise::memoise and promises::future_promise within a plumber endpoint?

How to combine future_promise with memoise in R plumber

0 Answers0