0

I'm trying to scrape some images on google. and I found this code, but it doesn't work. How can I fix this error??

from urllib.request import urlopen
from urllib.request import urlretrieve
from urllib.parse import quote_plus
from bs4 import BeautifulSoup
from selenium import webdriver

search= input('검색어:')
url = f'https://www.google.com/search?q={quote_plus(search)}&source=inms&tbm=isch&sa=X&ved=2haUKEwid64aF87LoAhUafd4KHcEtBZEQ_AUoAXoECBgQAw&biw=1536&bih=754'

driver = webdriver.Chrome()
driver.get(url)

html = driver.page_source
soup = BeautifulSoup(html)
img = soup.select('.rg_i.Q4LuWd.tx8vtf') 
n = 1
imgurl = []
for i in img:
    try:
        imgurl.append(i.attrs['src'])
    except KeyError:
        imgurl.append(i.attrs["data-src"])

for i in imgurl:
    urlretrieve(i,"크롤링 사진/"+ search + ".jpg")
    n +=1
    print(imgurl)

driver.close()

(검색어 means 'search term' in korean.)

error message

Traceback (most recent call last):
  File "C:\Users\u\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\common\service.py", line 72, in start
    self.process = subprocess.Popen(cmd, env=self.env,
  File "C:\Users\u\AppData\Local\Programs\Python\Python38-32\lib\subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Users\u\AppData\Local\Programs\Python\Python38-32\lib\subprocess.py", line 1307, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] 지정된 파일을 찾을 수 없습니다

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "yt.py", line 10, in <module>
    driver = webdriver.Chrome()
  File "C:\Users\u\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 73, in __init__    self.service.start()
  File "C:\Users\u\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\common\service.py", line 81, in start    raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

Thank you for your help.


노지우
  • 5
  • 1
  • I assume it is the second issue: "selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home " – Nico Müller Jun 03 '20 at 18:13
  • Does this answer your question? [Error message: "'chromedriver' executable needs to be available in the path"](https://stackoverflow.com/questions/29858752/error-message-chromedriver-executable-needs-to-be-available-in-the-path) – DaVinci Jun 03 '20 at 18:37

2 Answers2

0

It looks like you're missing chromedriver.exe or if you have installed it it's not on your path. Chromedriver is necessary to run Selenium.

Follow the instructions here to add it...

Download chromedriver: https://chromedriver.chromium.org/

Add it to path: https://developers.refinitiv.com/sites/default/files/How%20To%20Add%20ChromeDriver%20To%20System%20Variables_0.pdf

Aziz Sonawalla
  • 2,482
  • 1
  • 5
  • 6
0

Install webdriver-manager

pip install webdriver-manager


from urllib.request import urlopen
from urllib.request import urlretrieve
from urllib.parse import quote_plus
from bs4 import BeautifulSoup
from selenium import webdriver
# importing the installed package
from webdriver_manager.chrome import ChromeDriverManager

search= input('검색어:')
url = f'https://www.google.com/search?q={quote_plus(search)}&source=inms&tbm=isch&sa=X&ved=2haUKEwid64aF87LoAhUafd4KHcEtBZEQ_AUoAXoECBgQAw&biw=1536&bih=754'

                          # this will install chromedriver manager 
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get(url)

html = driver.page_source
soup = BeautifulSoup(html)
img = soup.select('.rg_i.Q4LuWd.tx8vtf') 
n = 1
imgurl = []
for i in img:
    try:
        imgurl.append(i.attrs['src'])
    except KeyError:
        imgurl.append(i.attrs["data-src"])

for i in imgurl:
    urlretrieve(i,"크롤링 사진/"+ search + ".jpg")
    n +=1
    print(imgurl)

driver.close()
DaVinci
  • 868
  • 1
  • 7
  • 25