2

I am trying to generate a scrip that downloads a file from the first link of this set of links here "Cleveland Fed Estimates of Inflation Expectations" from the download icon.

I know how to get a file with a permanent url, but in this case the file will be updated regularly and the URL as well as the file name changes frequently.

What is the best way to approach this?

Wolf
  • 562
  • 1
  • 7
  • 19
  • 1
    This isn't necessarily an R problem. I would just use another scripting language with better support for web scraping (e.g., Python + BeautifulSoup or PyQuery). – Alex P. Miller Jul 02 '15 at 13:44
  • 2
    @alexpmil you may want to double-check your facts. R excels at web scraping. `httr`, `rvest`, `xml2`, `XML` – hrbrmstr Jul 02 '15 at 14:10
  • 2
    @hrbrmstr Great to know. That hasn't always been the case. I suppose the best answer would be "use whichever language you are most comfortable web scraping with". – Alex P. Miller Jul 02 '15 at 15:36

1 Answers1

1

Not my area, but I'd suggest you build an API or something similar, to start, take a look here, here and here.

You'd need to get a key from the F(R)ED. Not sure whether the Cleveland one has something like this, but St. Louis certainly does.

Take a look at quantmod, as well, it is available as a package from CRAN, another option is FredR, which you would need to get from Github.

Community
  • 1
  • 1
erasmortg
  • 3,246
  • 1
  • 17
  • 34
  • Thank you. I am working with quantmod. On a realated note, could you do me a favor and check if this will work for you. It has been giving me an error all morning and worked fine yesterday. `library(quantmod) getSymbols("GDPC1",src="FRED")` – Wolf Jul 02 '15 at 14:07
  • 2
    `Error in download.file(paste(FRED.URL, "/", Symbols[[i]], "/", "downloaddata/", : cannot open URL 'http://research.stlouisfed.org/fred2/series/GDPC1/downloaddata/GDPC1.csv'` (but that URL works fine when I paste into a browser) – hrbrmstr Jul 02 '15 at 14:11