1

i working on a script for scraping video titles from this webpage " https://www.google.com.eg/trends/hotvideos " but the proplem is the titles are hidden on the html source page but i can see it if i used the inspector to looking for that

that's my code it's working good with this ("class":"wrap") but when i used that with the hidden one like "class":"hotvideos-single-trend-title-container" that's did't give me anything on output

#import urllib2
from urllib.request import urlopen
from bs4 import BeautifulSoup
html = urlopen('https://www.google.com.eg/trends/hotvideos').read()
soup = BeautifulSoup(html)
print (soup.findAll('div',{"class":"hotvideos-single-trend-title-container"}))
#wrap
Mohamed Sharkwy
  • 33
  • 1
  • 9

1 Answers1

0

The page is generated/populated by using Javascript.

BeautifulSoup won't help you here, you need a library which supports Javascript generated HTML pages, see here for a list or have a look at Selenium

Community
  • 1
  • 1
Maximilian Peters
  • 30,348
  • 12
  • 86
  • 99