Processing multiple quant mod stock tickers in parallel in R

Question

I might not explain my questions clear in the title, apologize. Here is the question with the code, it will be clearer,

library(zoo);library(quantmod)
stockData <- new.env()#Make a new environment for quantmod to store data in
startDate = as.Date("2006-12-29") #Specify period of time we are interested in
endDate = as.Date("2012-12-31")
monthlyStartDate = as.Date("2007-01-01")

# tickers <- c("AAPL","GOOG", "IBM")
tickers <- c("AAPL","GOOG", "IBM", "MSFT", "INTC", "YHOO", "F", "GS", "UL") 
# The tickers vector could be even larger, i.e.  50 stocks

stockData$AAPL.ret=diff(log(stockData$AAPL$AAPL.Adjusted)) # Compute log returns
stockData$GOOG.ret=diff(log(stockData$GOOG$GOOG.Adjusted)) # Compute log returns
stockData$IBM.ret=diff(log(stockData$IBM$IBM.Adjusted)) # Compute log returns
head(stockData$GOOG.ret)
head(stockData$GOOG$GOOG.Adjusted)

AAPLmonthly<-aggregate.zoo(stockData$AAPL.ret[2:nrow(stockData$AAPL$AAPL.Adjusted),],as.yearmon,sum)
GOOGmonthly<-aggregate.zoo(stockData$GOOG.ret[2:nrow(stockData$GOOG$GOOG.Adjusted),],as.yearmon,sum)
IBMmonthly<-aggregate.zoo(stockData$IBM.ret[2:nrow(stockData$IBM$IBM.Adjusted),],as.yearmon,sum)


head(AAPLmonthly)
stockret = cbind(AAPLmonthly, GOOGmonthly, IBMmonthly)
head(stockret)

The above code only used 3 tickers as an example, I want to know how to do a loop in R to make my whole vector of tickers into the stockret zoo object, could anybody help me? Much appreciated.

I am learning environment, just learnt the function is an closure, including the body, arguments and its environment, but I didn't know we could new an environment. So I got stuck here, stockData$AAPL.ret how do I put the stockData$ in front of my each element while doing a loop to assign values? Should I use "[" to do this? Besides, if I use the assign function to do this, in the code

stockData$AAPL.ret=diff(log(stockData$AAPL$AAPL.Adjusted)) # Compute log returns

how to do this, I'm just confused about how to make this stockData$AAPL$AAPL.Adjusted a more general argument in my assign function, any example would be much appreciated!

Answer 1

You can use get to get data from an environment and assign to assign data to a symbol in an environment.

library(quantmod)
stockData <- new.env()
tickers <- c("AAPL","GOOG", "IBM", "MSFT", "INTC", "YHOO", "F", "GS", "UL") 
getSymbols(tickers, src="yahoo", env=stockData)

for (tick in tickers) {
  x <- get(tick, pos=stockData)  # get data from stockData environment
  x$ret <- diff(log(Ad(x)))      # add a column with returns
  assign(tick, x, pos=stockData) # assign back into stockData environment
  assign(paste0(tick, "monthly"), 
         apply.monthly(x, sum, na.rm=TRUE), 
         pos=stockData) # calc monthly sum and assign in stockData environment
}

Or, your could use eapply to apply a function to every object in the environment The result will be a list that you can coerce back into an environment

stockData <- as.environment(eapply(stockData, function(x) {
  x$ret <- diff(log(Ad(x)))
  x
}))

Or you could create a list of only the returns, then loop over that to calculate the monthly sums

R <- eapply(stockData, function(x) diff(log(Ad(x))))
monthly <- lapply(R, apply.monthly, sum, na.rm=TRUE)

You can merge the results into a single object like this

do.call(merge, R)
do.call(merge, monthly)

Answer 2

Assignment outside the local scope

Firstly, function bodies operate in a child scope, even when assigning to variables declared in the global or containing scope.

R has a special operator <<- for setting variables in parent scopes (environments). See ?assignOps for details.

For example:

a <- 1
(function(){ a <- a + 1 })()

A new a is created in the local scope at the point of assignment, so the internal value goes to 2, but the global remains 1.

a <- 1
(function(){ a <<- a + 1 })()

Using the <<- assignment operator searches the parent environments in turn and finds our global a. Global A gets the value 2 in this case.

Functional Programming

In R, functional programming is encouraged. See my other answer on quantmod for how you might approach this with time series.

Answer 3

Looks like a job for lapply and do.call for me, for instance (untested)

tickers <- c("AAPL","GOOG", "IBM")

as_monthly <- function(sym) {
    log_returns <- diff(log(stockdata[[sym]][[paste(sym, "Adjusted", sep=".")]]))
    aggregate(log_returns[2:nrow(log_returns),], as.yearmon, sum)
}

assign("stockret", do.call(cbind, lapply(tickers, as_monthly)), envir=stockdata)

It is not clear from your question how you populate the stockdata environment, though.

Answer 4

Quantmod has the unusual behavior that it adds its results to the enclosing environment.

> getSymbols(c("AAPL","ORCL"))
[1] "AAPL" "ORCL"

returns the list of symbols it retrieves, but has the side effect that it assigns the results into the caller's environment:

ls()
[1] "AAPL" "ORCL"

> nrow(AAPL)
1774
> class(AAPL)
[1] "xts" "zoo"

For writing scripts, this can be messy, so as the OP suggests it can be a good idea to create a dedicated environment for this:

> stocks <- new.env()
> getSymbols(c("AAPL","ORCL"),stocks)
> ls(stocks)
[1] "AAPL" "ORCL"

This approach prevents name collisions, and allows the returned results to be processed as a group. Now we have collected the data together, it's more convenient to replace the environment with a more traditional data structure, for instance, have each ticker be a list element:

> stocksL <- as.list(stocks)

Now we can use Map or Lapply effectively.

> Map(function() { "do something " }, stocksL)

Note that since getSymbols prefixes the column names with the ticker name, it would be useful to clean that off:

> stocksL2 <- Map(function(x) {names(x)<-gsub(".*\\.","",names(x));x}, stocksL)

Now we can reprocess it

> adj <- Map(function(x) {diff(log(x$Adjusted))}, stocksL2)