Use this tag for questions specific to the future ecosystem in R - typically used for parallel and asynchronous processing. At the core of this ecosystem is Future API and the future package, which a larger set of packages then build upon.
Questions tagged [r-future]
94 questions
7
votes
1 answer
shiny app download button and future_promise: Error in enc2utf8: argument is not a character vector
I am trying to create a download handler in shiny, but using future_promise() because it is possible that writing the file could take some time. Here is a working example of what I'd like to do, but without using the async framework:
A working .Rmd…
richarddmorey
- 976
- 6
- 19
7
votes
0 answers
memory not being released in future with nested plan(multisession)
Recently I've been playing with doing some parallel processing in R using future (and future.apply and furrr) which has been great mostly, but I've stumbled onto something that I can't explain. It's possible that this is a bug somewhere, but it may…
seth127
- 2,594
- 5
- 30
- 43
6
votes
1 answer
terra package returns error when try to run parallel operations
I'm working with raster package and I try to switch to terra but for some reasons that I don't understand, terra cannot reproduce the same operation of raster when working in parallel with packages such snowfall and future.apply. Here is a…
Elia
- 2,210
- 1
- 6
- 18
6
votes
1 answer
Kill futures from future_apply() on Linux
I use future_lapply() to parallel my code on a Linux machine. If I terminate the process early, only one worker is freed and the parallel processes continue to persist. I know I can enter tools::pskill(PID) to end each individual process, but this…
Mxblsdl
- 484
- 6
- 16
6
votes
1 answer
furrr doesn't find own package
I am currently working on a package, let's say it is called myPack.
I have a function called myFunc1 and another function called myFunc2, which looks something like that:
myFunc2 <- function(x, parallel = FALSE) {
if(parallel)…
user11143533
- 79
- 3
4
votes
1 answer
Use future and promise inside reactive R shiny
I have a question regarding R shiny and more especially about the way to improve performance of my shiny app. I'm using some SQL queries that take quite long to run as well as some plot which take time to show up. I know that it is possible to use…
CCbs
- 105
- 3
4
votes
0 answers
R future package - understanding error message
I am submitting two jobs as below on Linux (R 3.6.0). See below sample code.
library("future")
plan(multicore)
job1 <- future({
print("job1")
})
job2 <- future({
print("job2")
})
while (!resolved(job1)) {
Sys.sleep(0.2)
}
while…
R007
- 101
- 1
- 13
4
votes
1 answer
R future multisession limiting number of cpus
I'm currently parallellising by using the future package as follows:
plan(multisession, gc = TRUE)
standardised_addresses1 <- future_lapply(1:20000, function(x) x*x)
The problem is that it uses all the CPUs on the server.
I would like to limit the…
Dario Federici
- 1,228
- 2
- 18
- 40
3
votes
0 answers
How to call system() within %dopar% iterations in R
How should I call external programs from sub-instances of parallelized R? The problem could occur also on other contexts, but I am using library(foreach) and library(doFuture) on slurm-based HPC. As an example, I have created a hello.txt that…
Imsa
- 69
- 4
3
votes
0 answers
`doParallel` vs `future` while using `Seurat` package
Here is the story.
From Seurat vignette, FindMarkers() can be accelerated by utilizing future package, future::plan("multiprocess", workers = 4)
However, I am running a simulation that I need to use FindAllMarkers() inside a doParallel::foreach()…
yuw444
- 380
- 2
- 10
3
votes
1 answer
Why do withr::with_seed and R.utils::withSeed generate different results when used in async code?
I'm trying to learn async programming using R so that I can implement an app that requires generation of random numbers with specified seeds (always with specified seeds). I've been using R.utils::withSeed for this, but I know that withr::with_seed…
richarddmorey
- 976
- 6
- 19
3
votes
1 answer
Do I need to use future_map or map to parallelize fable forecast?
I've created a tsibble of ~75K time series in R Studio on my local machine.
I'm looking for ways to speed up the processing time before I migrate the process to a VM with more processing power.
Does Fable handle all of the parallel processing in the…
Barrett Layman
- 31
- 1
3
votes
1 answer
Is there a way to modify the cluster resources allocated futures when using a furrr function?
I'm using the future, future.batchtools and furrr packages within my R script, in order to run code on a slurm-managed remote machine.
When I define my future topology, I pass a template file to future.bacthtools::batchtools_slurm() and define the…
Johannes Koch
- 71
- 2
3
votes
0 answers
Is there a better way to use disk.frame within a function?
I have created some functions that need to handle either a disk.frame or a data.table as input. I am getting errors from the future package used within disk.frame due to an object not being found upon execution. I think this is due to the fact that…
pmbrophy
- 31
- 2
3
votes
0 answers
How to process knitr code chunks asynchronously?
I'm wondering if there's a way to process knitr code chunks in an RMarkdown document asynchronously.
Here's what I have in mind: imagine a document containing a complex data analysis broken down into several code chunks, several of which contain…
Dr. Andrew John Lowe
- 510
- 8
- 20