4

I am using BeautifulSoup scraping content on youtube video list. But the thumbnails load by JS. So I can't get them if i use BeautifulSoup only.

I found some articles told me selenium could do this.

I installed chromedriver

root@localhost:~# chromedriver -v
ChromeDriver 2.33.506092 (733a02544d189eeb751fe0d7ddca79a0ee28cce4)

here are my python code

import time

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

browser = webdriver.Chrome()

browser.get("https://www.youtube.com/results?search_query=xxx")
time.sleep(1)

elem = browser.find_element_by_tag_name("body")

no_of_pagedowns = 20

while no_of_pagedowns:
    elem.send_keys(Keys.PAGE_DOWN)
    time.sleep(0.2)
    no_of_pagedowns -= 1

post_elems = browser.find_elements_by_class_name("branded-page-box")

for post in post_elems:
    print post.text

then I got error

Traceback (most recent call last):
  File "youtubelist.py", line 68, in <module>
    browser = webdriver.Chrome('/usr/local/bin/chromedriver')
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/chrome/webdriver.py", line 69, in __init__
    desired_capabilities=desired_capabilities)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 151, in __init__
    self.start_session(desired_capabilities, browser_profile)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 240, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 308, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot find Chrome binary
  (Driver info: chromedriver=2.33.506092 (733a02544d189eeb751fe0d7ddca79a0ee28cce4),platform=Linux 4.9.36-x86_64-linode85 x86_64)

Does anyone know how to solve this error? thank you!

jerry
  • 319
  • 1
  • 5
  • 13

0 Answers0