Exporting FinancialInstrument:::.instrument to foreach workers and unpacking it

Question

I am doing some parallel processing and need to access instrument properties from FinancialInstrument:::.instrument environment at each worker spawned by parallel processing.

Simple instrument.list <- as.list(FinancialInstrument:::.instrument) and using .export argument with foreach does not work (as the code below shows it works when no parallel backends are registered and does not when they are). Please see reproducible example below:

library(FinancialInstrument)
library(parallel)
library(doParallel)
library(foreach)

currency(c("USD", "EUR")) # define some currencies
stock(c("SPY", "LQD", "IBM", "GS"), currency="USD") # define some stocks
exchange_rate("EURUSD") # define an exchange rate

ls_stocks() #get the names of all the stocks
ls_instruments() # all instruments

getInstrument("IBM")

instrument.list <- as.list(FinancialInstrument:::.instrument)

# First let's run this without parallel backend registered to show the code works
foreach(1:2,
        .packages="FinancialInstrument",
        .export="instrument.list"
        ) %dopar% {
            .instrument <- as.environment(instrument.list)
            message(paste0("Available instruments in .instrument environment: ", paste(ls_instruments(), collapse=", "), " ."))
        }


# Now, let's register a parallel backend
cl <- makeCluster(2, outfile="log.txt")
registerDoParallel(cl)

# And by looking in log.txt file we see that .instrument environment is not functioning properly. How to make this work?
foreach(1:2,
        .packages="FinancialInstrument",
        .export="instrument.list"
        ) %dopar% {
            .instrument <- as.environment(instrument.list)
            message(paste0("Available instruments in .instrument environment: ", paste(ls_instruments(), collapse=", "), " ."))
        }

stopCluster(cl)

score 1 · Accepted Answer · answered Jan 02 '17 at 10:16

I got this working using assignInNamespace function in worker. Below is now a working code from the question above:

library(FinancialInstrument)
library(parallel)
library(doParallel)
library(foreach)

currency(c("USD", "EUR")) # define some currencies
stock(c("SPY", "LQD", "IBM", "GS"), currency="USD") # define some stocks
exchange_rate("EURUSD") # define an exchange rate

ls_stocks() #get the names of all the stocks
ls_instruments() # all instruments

getInstrument("IBM")

instrument.list <- as.list(FinancialInstrument:::.instrument)

# First let's run this without parallel backend registered to show the code works
foreach(1:2,
        .packages="FinancialInstrument",
        .export="instrument.list"
        ) %dopar% {
            .instrument <- as.environment(instrument.list)
            message(paste0("Available instruments in .instrument environment: ", paste(ls_instruments(), collapse=", "), " ."))
        }


# Now, let's register a parallel backend
cl <- makeCluster(2, outfile="log.txt")
registerDoParallel(cl)

# And by looking in log.txt file we see that .instrument environment is not functioning properly. How to make this work?
foreach(1:2,
        .packages="FinancialInstrument",
        .export="instrument.list"
        ) %dopar% {
            assignInNamespace(".instrument", as.environment(instrument.list), "FinancialInstrument")
            message(paste0("Available instruments in .instrument environment: ", paste(ls_instruments(), collapse=", "), " ."))
        }

stopCluster(cl)

Exporting FinancialInstrument:::.instrument to foreach workers and unpacking it

1 Answers1