0

The following PHP code (on Laravel) has been working for months until today, where I am now receiving a 502 error indicating connection refused:

$url = "https://query.yahooapis.com/v1/public/yql?q=select+%2A+from+yahoo.finance.quotes+where+symbol+in+%28%27YHOO%27%29&format=json&env=%20store%3A%2F%2Fdatatables.org%2Falltableswithkeys&callback=";

$data = file_get_contents($url);
dd($data);

When I login to the Yahoo Developer Console to test, I'm able to query the public datatables just fine. However, when I attempt to use the console to access any of the community datatables like this, I am returned the following (this error occurs with all community datatables):

{
 "error": {
  "lang": "en-US",
  "diagnostics": {
   "url": {
    "execution-start-time": "0",
    "execution-stop-time": "7",
    "execution-time": "7",
    "http-status-code": "502",
    "http-status-message": "Connection refused",
    "content": "http://www.datatables.org/yahoo/finance/yahoo.finance.quotes.xml"
   }
  },
  "description": "No definition found for Table yahoo.finance.quotes"
 }
}

Using a different machine/IP address, I am able to access these community tables just fine as long as I remain anonymous. Once I login to the Yahoo Developer Console, the Connection refused message appears again.

I suspect it has something to do with Yahoo possibly blacklisting my IP address or throttling my account as I run this script for every stock (3,000) once per day.

I attempted to access the URL referenced in the error message content node and it appears this site is down. I'm not sure if this content node matters to the API, as I am able to access it from a different IP address.

I'm open to suggestions and recommendations on how I can determine/fix this problem.

Nakilon
  • 34,866
  • 14
  • 107
  • 142
Gunnar
  • 661
  • 1
  • 11
  • 29
  • This is the second long (24h+) outage at http://www.datatables.org this month :( Not sure what the core problem is here, or if these outage are set to continue. It would be nice if someone connected with 'datatables.org' could let us know. – user3416119 Mar 16 '16 at 14:11

2 Answers2

4

Yahoo Community tables are stored in datatables.org, and datatables.org is down; that's why there's a querying problem.

http://www.datatables.org

Peter O.
  • 32,158
  • 14
  • 82
  • 96
Oussama
  • 595
  • 5
  • 16
  • Thanks. Very relieved to hear that that's the reason. Any idea who maintains datatables.org and why they are down? – jco40 Mar 16 '16 at 10:11
  • It's a yahoo domain see http://www.whois.com/whois/datatables.org You can use this service instead: http://fixer.io/ but it doesnt contain all currencies – Oussama Mar 16 '16 at 10:17
  • datatables.org is up now, and all community tables are accessible. – Gunnar Mar 16 '16 at 16:38
0

In one of the forum posts, a Yahoo employee has confirmed that the free EOD data has been terminated, and will not be reintroduced. Check out this thread and look for reply from Nixon. Yahoo is recently acquired by Verizon, and it must be the new direction.

However, if you check the Yahoo financial page, the CSV download link works, though differently now. It is through a new API that uses an authentication token "crumb" that is linked to a cookie when you access the page.

I have put together some quick Python3 code to download the same CSV as before through this new API. Please check out the GitHub project yahoo_quote_download.

c0redumb
  • 371
  • 4
  • 5
  • I saw that too, and thanks for your git example. I wound up simply crawling and extracting the table based on the search criteria like this: https://finance.yahoo.com/quote/MSFT/history?period1=1462863600&period2=1495436400&interval=1d&filter=history&frequency=1d and the xpath: //*[@id="quote-leaf-comp"]/section/div[2]/table – Gunnar May 22 '17 at 15:20
  • That would work too. It is too bad that Yahoo has discontinued a very useful feature. – c0redumb May 22 '17 at 15:50