0

I hacked together some pretty simple code, which I thought should allow me to screen scrape various tables from a URL. Here is my code.

import pandas as pd
import requests
from bs4 import BeautifulSoup

res = requests.get("https://www.federalreserve.gov/releases/h8/current/default.htm")
soup = BeautifulSoup(res.content,'lxml')
table = soup.find_all('table')[1]
df = pd.read_html(str(table))
print(df)

I am trying to grab data from table number 1, but when I run the code I get this error:

IndexError: list index out of range

I would like to enter a table number and get the relevant data from that specific table. There are a total of 11 tables on the page. What am I doing wrong?

ASH
  • 20,759
  • 19
  • 87
  • 200
  • 1
    I found a solution that works! https://stackoverflow.com/questions/10556048/how-to-extract-tables-from-websites-in-python – ASH Sep 11 '20 at 22:59

1 Answers1

0

I believe you just want a standard html parser rather than using lxml.

Change:

soup = BeautifulSoup(res.content,'lxml')

To:

soup = BeautifulSoup(res.content,'html.parser')

Output:

[        Account                                                        2019 Aug     2020 Feb     2020 Mar     2020 Apr     2020 May     2020 Jun     2020 Jul     2020 Aug  Week ending                                       
        Account                                          Account.1     2019 Aug     2020 Feb     2020 Mar     2020 Apr     2020 May     2020 Jun     2020 Jul     2020 Aug       Aug 12       Aug 19       Aug 26       Sep 02
0        Assets                                             Assets       Assets       Assets       Assets       Assets       Assets       Assets       Assets       Assets       Assets       Assets       Assets       Assets
1             1                                        Bank credit      13606.2      13949.9      14419.2      14787.2      14851.9      14840.1      14880.2      14889.9      14878.1      14873.9      14893.1      14912.1
2             2                        Securities in bank credit 2       3725.5       3871.0       3987.4       3989.3       4016.8       4166.5       4297.2       4363.6       4327.7       4357.0       4375.1       4421.3

....

jwjhdev
  • 195
  • 7