Getting a CSS table with beautiful soup

Question

I have tried to get the data from this table but I have been unable to do so: https://datagolf.ca/player-trends

I have tried many things for the last few hours, below is my most recent when just returns an empty list.


import bs4
import requests

res = requests.get('https://datagolf.ca/player-trends')

soup = bs4.BeautifulSoup(res.text, 'lxml')

table = soup.find_all("div", class_ = "table")

table

Is the issue something similar to this:

Scrape of table with only 'div's

You probably need to use selenium to do this. – Rakshith G B Jun 24 '19 at 16:50 — Rakshith G B, Jun 24 '19 at 16:50

score 0 · Answer 1 · answered Jun 24 '19 at 16:48

0

This page is java script rendered. Take a closer look at what requests.get('https://datagolf.ca/player-trends') actually returns. It does not contain the table.

Despite that it pulls the data from https://dl.dropboxusercontent.com/s/hprrfklqs5q0oge/player_trends_dev.csv?dl=1

answered Jun 24 '19 at 16:48

Raphael

1,731
2
7
23

Magic thanks I have just taken that straight into a dataframe – sinovius Jun 24 '19 at 17:21

Getting a CSS table with beautiful soup

1 Answers1