getSymbols not returning data as expected

Question

I'm having some trouble getting equity pricing data into R. I have a list of over 4k symbols, and some are no longer active/ valid. I was using an lapply with get.hist.quote on the list, but some bad tickers stopped it in its tracks too much for it to be a practical approach. Following this post: l_ply: how to pass the list's name attribute into the function? and attempting to use the accepted answer is the seemingly best approach I can find, but it isn't working. The l_ply doesn't load any symbols into my workspace. I get these warnings:

Warning messages:
1: In download.file(paste(yahoo.URL, "s=", Symbols.name, "&a=", from.m,  :
  downloaded length 90279 != reported length 200
2: In download.file(paste(yahoo.URL, "s=", Symbols.name, "&a=", from.m,  :
  downloaded length 87827 != reported length 200
3: In download.file(paste(yahoo.URL, "s=", Symbols.name, "&a=", from.m,  :
  downloaded length 89737 != reported length 200

The code from the post cited above that I'm trying to use is below. Is there a fix to this? I'm seeing other postings regarding similar situations, but fixes such as this one aren't working for me. If anyone else has a good way for me to get daily data on many tickers, I'm open to all suggestions. Thanks in advance.

library(quantmod)
library(plyr)
symbols <- c("MSFT","C","MMM")

#1
l_ply(symbols, function(sym) try(getSymbols(sym))) 
symbols <- symbols[symbols %in% ls()]

#2
sym.list <- llply(symbols, get) 

#3
data <- xts()
for(i in seq_along(symbols)) {
  symbol <- symbols[i]
  data <- merge(data, get(symbol)[,paste(symbol, "Close", sep=".")])
}

Those are warnings, not errors. But, you say "not returning data as expected." What was expected and how is it different than what you got? — GSee, Mar 03 '14 at 02:09
Poor wording; yes, warnings. I was expecting symbols with data in my workspace; but, I have none. — StatsViaCsh, Mar 03 '14 at 02:15
Oh I see. If you used `getSymbols(sym, env=globalenv())` it'd do what you expected. I'll add an answer with the more common idiom — GSee, Mar 03 '14 at 02:18
a couple related posts: http://stackoverflow.com/questions/11179154/r-names-of-quantmod-variables, http://stackoverflow.com/questions/5574595/getsymbols-and-using-lapply-cl-and-merge-to-extract-close-prices?lq=1 — GSee, Mar 03 '14 at 02:33

GSee · Accepted Answer · 2014-03-12T22:34:19.790

The issue is that by default getSymbols assigns the data in the parent.frame(). When you call getSymbols() from a function, the parent.frame() is no longer the .GlobalEnv that you're used to. So, a quick fix to your current could would be to replace

l_ply(symbols, function(sym) try(getSymbols(sym)))

with

l_ply(symbols, function(sym) try(getSymbols(sym, env=globalenv())))

Note that you may run into trouble at the next step where you use get. You should probably also specify which environment you want to get the data from so that it would also work if you put it in a function. i.e. sym.list <- llply(symbols, get, pos=globalenv())

Here's a more common idiom for getting and merging data with quantmod

library(quantmod)
s <- c("MSFT","C","MMM")
e <- new.env() #environment in which to store data
getSymbols(s, src="yahoo", env=e)
do.call(merge, eapply(e, Cl)[s])

Or, using try like the OP

L <- lapply(symbols, function(sym) try(getSymbols(sym, auto.assign=FALSE)))
do.call(merge, lapply(L, Cl))

getSymbols not returning data as expected

1 Answers1