Python scrape dynamic table from website?

Question

I'm trying to get info from a table which values change given some inputs (you have to choose the date of analysis)

I've never get info from a "dynamic" table so I haven't got anywhere

from bs4 import BeautifulSoup

from urllib.request import urlopen, Request

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.3'}

reg_url = "http://www.cmfchile.cl/institucional/mercados/entidad.php?mercado=V&rut=9117&grupo=&tipoentidad=FIRES&row=AAAw%20cAAhAABP4LAAU&vig=VI&control=svs&pestania=7"

req = Request(url=reg_url, headers=headers) 

html = urlopen(req).read() 

soup = BeautifulSoup(html)

This is quite a broad question. Is that code all you’ve tried? — AMC, Nov 11 '19 at 21:09

score 0 · Answer 1 · answered Nov 11 '19 at 21:14

I think that you can't get dynamic data with BeautifulSoup.

You should better use Selenium for this, you can change your dates to whatever you want, then get any elements that you will need.

Check this out: How to retrieve the values of dynamic html content using Python

Python scrape dynamic table from website?

1 Answers1