Python - Pulling from Google Finance with Pandas

Question

I'm trying to pull data from Google Finance using Pandas and Pandas Datareader. Here is my code:

#Importing libraries needed for pulls from Google
from pandas_datareader import data
import pandas as pd
import datetime
from datetime import date

#Define the instruments to download.  In this case: Apple, Microsoft, and 
the S&P500 index
tickers = ['APPL', 'MSFT', 'SPY']
start_date = datetime.datetime(2017, 12, 1)
end_date = datetime.datetime(2017, 12, 31)

#Use pandas_reader.data.DataReader to load the desired data
panel_data = data.DataReader('SPY', 'google', start_date, end_date)
#Getting just the adjusted closing prices.  This will return a Pandas DataFrame
#The index in this DataFrame is the major index of the panel_data.
close = panel_data.ix['Close']

#Getting all weekdays within date range.
all_weekdays = pd.date_range(start=start_date, end=end_date, freq='B')

#How do we align the existing prices in the adj_close with out new set of dates?
#All we need to do is reindex close using all_weekdays as the new index.
close = close.reindex(all_weekdays)

close.head(10)

And here is the console output:

runfile('C:/Users/kjohn_000/.spyder-py3/temp.py', wdir='C:/Users/kjohn_000/.spyder-py3')
C:\Users\kjohn_000\Anaconda3\lib\site-packages\pandas_datareader\base.py:201: SymbolWarning: Failed to read symbol: 
'APPL', replacing with NaN.
  warnings.warn(msg.format(sym), SymbolWarning)
C:\Users\kjohn_000\Anaconda3\lib\site-
packages\pandas_datareader\base.py:201: SymbolWarning: Failed to read 
symbol: 'MSFT', replacing with NaN.
  warnings.warn(msg.format(sym), SymbolWarning)
C:\Users\kjohn_000\Anaconda3\lib\site-packages\pandas_datareader\base.py:201: SymbolWarning: Failed to read symbol: 
'SPY', replacing with NaN.
  warnings.warn(msg.format(sym), SymbolWarning)
Traceback (most recent call last):

  File "<ipython-input-2-0ddd75de0396>", line 1, in <module>
    runfile('C:/Users/kjohn_000/.spyder-py3/temp.py', 
wdir='C:/Users/kjohn_000/.spyder-py3')

  File "C:\Users\kjohn_000\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
    execfile(filename, namespace)

  File "C:\Users\kjohn_000\Anaconda3\lib\site-
packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/Users/kjohn_000/.spyder-py3/temp.py", line 14, in <module>
    panel_data = data.DataReader(tickers, dataSource, start_date, end_date)

  File "C:\Users\kjohn_000\Anaconda3\lib\site-packages\pandas_datareader\data.py", line 137, in DataReader
session=session).read()

  File "C:\Users\kjohn_000\Anaconda3\lib\site-
packages\pandas_datareader\base.py", line 186, in read
    df = self._dl_mult_symbols(self.symbols)

  File "C:\Users\kjohn_000\Anaconda3\lib\site-
packages\pandas_datareader\base.py", line 206, in _dl_mult_symbols
    raise RemoteDataError(msg.format(self.__class__.__name__))

RemoteDataError: No data fetched using 'GoogleDailyReader'

Why is Pandas Datareader failing to read the stock symbols in the 'tickers' list? I've looked around for answers for a few hours now, but many of the answers were in response questions regarding the Yahoo API, and the remaining answers were either for another language or simply out of my depth as coding goes (I'm relatively new to Python). Thanks in advance for help and feedback.

[This github issue](https://github.com/pydata/pandas-datareader/issues/394) is a recent issue not related to Yahoo. Perhaps it's ongoing during the Google restructure of their API? Or you may need to update pandas. — roganjosh, Feb 20 '18 at 23:00
I see. Very helpful link. My version of Pandas is up to date as far as Anaconda knows, but the url it tries to call is 'https://www.google.com/finance/historical' instead of 'http://finance.google.com/finance/historical'. The latter url is correct as of the API restructure. Path to the files to alter for Anaconda users is Anaconda3\Lib\site-packages\pandas_datareader\google\. Thanks for the help! — MakoTheShark, Feb 21 '18 at 00:06

score 1 · Answer 1 · answered Feb 21 '18 at 05:16

This worked for me with Python 3.6.1

from pandas_datareader import data
import fix_yahoo_finance as yf
yf.pdr_override() 

symbol = 'AMZN'
data_source='google'
start_date = '2010-01-01'
end_date = '2016-01-01'
df = data.get_data_yahoo(symbol, start_date, end_date)
print(df)
df.head()

This worked for me as well.

from urllib.request import urlopen
from bs4 import BeautifulSoup as bs

def get_historical_data(name, number_of_days):
    data = []
    url = "https://finance.yahoo.com/quote/" + name + "/history/"
    rows = bs(urlopen(url).read()).findAll('table')[0].tbody.findAll('tr')

    for each_row in rows:
        divs = each_row.findAll('td')
        if divs[1].span.text  != 'Dividend': #Ignore this row in the table
            #I'm only interested in 'Open' price; For other values, play with divs[1 - 5]
            data.append({'Date': divs[0].span.text, 'Open': float(divs[1].span.text.replace(',',''))})

    return data[:number_of_days]

#Test
for i in get_historical_data('googl', 25):   
    print(i)

I know this is a good long while after the fact, but I got my code working with a URL update, then a month or so later, it stopped working again without any changes being made. I've since started using the Morningstar API instead, and haven't had any issues with it. Anyways, thanks for taking the time to reply. — MakoTheShark, Apr 27 '18 at 02:55
https://stackoverflow.com/questions/44048671/alternatives-to-the-yahoo-finance-api — ASH, Apr 27 '18 at 14:54

score 0 · Answer 2 · answered Jul 10 '18 at 21:26

This isn't using Google, however if you use the python YahooFinancials module, you can easily load financial data into pandas. YahooFinancials gets the financial data by hashing out the Relevant Yahoo Finance Page's datastore object, so it's pretty fast, well built, and doesn't rely on the old discontinued api, or a webdriver like a web scraper does. Data is returned in JSON.

$ pip install yahoofinancials

Usage Example:

from yahoofinancials import YahooFinancials
import pandas as pd

# Select Tickers and stock history dates
ticker = 'AAPL'
ticker2 = 'MSFT'
ticker3 = 'INTC'
index = '^NDX'
freq = 'daily'
start_date = '2012-10-01'
end_date = '2017-10-01'


# Function to clean data extracts
def clean_stock_data(stock_data_list):
    new_list = []
    for rec in stock_data_list:
        if 'type' not in rec.keys():
            new_list.append(rec)
    return new_list

# Construct yahoo financials objects for data extraction
aapl_financials = YahooFinancials(ticker)
mfst_financials = YahooFinancials(ticker2)
intl_financials = YahooFinancials(ticker3)
index_financials = YahooFinancials(index)

# Clean returned stock history data and remove dividend events from price history
daily_aapl_data = clean_stock_data(aapl_financials.get_historical_stock_data(start_date, end_date, freq)[ticker]['prices'])
daily_msft_data = clean_stock_data(mfst_financials.get_historical_stock_data(start_date, end_date, freq)[ticker2]['prices'])
daily_intl_data = clean_stock_data(intl_financials.get_historical_stock_data(start_date, end_date, freq)[ticker3]['prices'])
daily_index_data = index_financials.get_historical_stock_data(start_date, end_date, freq)[index]['prices']
stock_hist_data_list = [{'NDX': daily_index_data}, {'AAPL': daily_aapl_data}, {'MSFT': daily_msft_data}, {'INTL': daily_intl_data}]


# Function to construct data frame based on a stock and it's market index
def build_data_frame(data_list1, data_list2, data_list3, data_list4):
    data_dict = {}
    i = 0
    for list_item in data_list2:
        if 'type' not in list_item.keys():
            data_dict.update({list_item['formatted_date']: {'NDX': data_list1[i]['close'], 'AAPL': list_item['close'],
                                                            'MSFT': data_list3[i]['close'],
                                                            'INTL': data_list4[i]['close']}})
            i += 1
    tseries = pd.to_datetime(list(data_dict.keys()))
    df = pd.DataFrame(data=list(data_dict.values()), index=tseries,
                      columns=['NDX', 'AAPL', 'MSFT', 'INTL']).sort_index()
    return df

score -1 · Answer 3 · answered Aug 19 '21 at 15:13

def stock(pape,dia): paper = pape dias = dia "Extract history from YF" url = "https://query1.finance.yahoo.com/v7/finance/download/" + paper + ".SA" + "?" + "period1=1597805534&period2=1629341534&interval=1d&events=history&includeAdjustedClose=true" base = paper + ".SA.csv" file= r'D:\Users\repo\' file2 = file + base if os.path.exists(file2): os.remove(file2) else: print("Not here") wget.download(url) base2 = pd.read_csv(base) pd.options.display.max_rows = 14000 #base2.tail(15) return base2.tail(dias)

Can you try reformatting this? Looks like there should be multiple lines. — mirthbottle, Aug 19 '21 at 22:32

Python - Pulling from Google Finance with Pandas

3 Answers3