I am dumping a webpage with requests, and I need to search for a string inside the code. I need to get the div fields and search for a certain value.
import requests, pprint
page = requests.get('')
tree = (page.content)
pp = pprint.PrettyPrinter(indent=4)
pp.pprint(tree)
You can use BeautifulSoup for such things - it's a HTML parser.
Refer more at: Beautiful Soup and extracting a div and its contents by ID
import requests, pprint import bs4
page = requests.get('')
tree = (page.content)
soup = bs4.BeautifulSoup('<html><body><div id="articlebody"> ... </div></body></html')
divs = soup.find_all("div")
texts = [i.text for i in soup.find_all("div")]
