Using bs4, how do I obtain content of elements that appear only after the dom has been manipulated?

Asked Jun 18 '18 at 09:43

Active Jun 18 '18 at 09:43

Viewed 45 times

I'm able to obtain a soup object by passing a html object like so:

from bs4 import BeautifulSoup as bs 

url = 'someurl.com'
html = urllib.request.urlopen(url)
soup = bs(html, "html.parser")

But the content of the element that I want does not appear until an action is taken on the page.
The soup that I obtain using this method does not have what I need.

What must I do to obtain an "updated" soup after the dom has been manipulated?

asked Jun 18 '18 at 09:43

Jay Jung

1,805
3
23
46

Possible duplicate of [Web-scraping JavaScript page with Python](https://stackoverflow.com/questions/8049520/web-scraping-javascript-page-with-python) – spejsy Jun 18 '18 at 09:47

Using bs4, how do I obtain content of elements that appear only after the dom has been manipulated?

0 Answers0