Yahoo Finance recently changed its historical data URLs from http to https. For example, here is the URL to obtain the maximum history for IBM (as of today, 2017-05-29):
https://query1.finance.yahoo.com/v7/finance/download/IBM?period1=-252345600&period2=1496041200&interval=1d&events=history&crumb=jIIyoewBT2o
When I try to read this resource from within R I get the following:
> url = 'https://query1.finance.yahoo.com/v7/finance/download/IBM?period1=-252345600&period2=1496041200&interval=1d&events=history&crumb=jIIyoewBT2o'
> read.csv(url)
Error in file(file, "rt") :
(converted from warning) cannot open URL 'https://query1.finance.yahoo.com/v7/finance/download/IBM?period1=-252345600&period2=1496041200&interval=1d&events=history&crumb=jIIyoewBT2o': HTTP status was '401 Unauthorized'
The download works fine when invoked manually through the browser, so it's not like Yahoo Finance is trying to prevent access to the data. Evidently, the browser is sending some secret sauce to the Yahoo server. Can I send the same secret sauce through an R script?