2

Well - i am trying to figure out the simplest approach to gather data from clutch.io

  • i have tried various approaches to gather data from a website (clutch.io ) but all seems to fail:

see here

from bs4 import BeautifulSoup
from selenium import webdriver

driver = webdriver.Chrome()

url = 'https://clutch.co/it-services/msp'

driver.get(url=url)
soup = BeautifulSoup(driver.page_source,"lxml")

links = []
for l in soup.find_all('li',class_='website-link website-link-a'):
    results = (l.a.get('href'))
    links.append(results)

print(links, "\n", "Count links - ", len(links))

throws back this error:

---------------------------------------------------------------------------
WebDriverException                        Traceback (most recent call last)
<ipython-input-4-4f37092106f4> in <cell line: 4>()
      2 from selenium import webdriver
      3 
----> 4 driver = webdriver.Chrome()
      5 
      6 url = 'https://clutch.co/it-services/msp'

5 frames
/usr/local/lib/python3.10/dist-packages/selenium/webdriver/remote/errorhandler.py in check_response(self, response)
    243                 alert_text = value["alert"].get("text")
    244             raise exception_class(message, screen, stacktrace, alert_text)  # type: ignore[call-arg]  # mypy is not smart enough here
--> 245         raise exception_class(message, screen, stacktrace)

WebDriverException: Message: unknown error: cannot find Chrome binary
Stacktrace:
#0 0x55a6ebf424e3 <unknown>
#1 0x55a6ebc71c76 <unknown>
#2 0x55a6ebc98757 <unknown>
#3 0x55a6ebc97029 <unknown>
#4 0x55a6ebcd5ccc <unknown>
#5 0x55a6ebcd547f <unknown>
#6 0x55a6ebcccde3 <unknown>
#7 0x55a6ebca22dd <unknown>
#8 0x55a6ebca334e <unknown>
#9 0x55a6ebf023e4 <unknown>
#10 0x55a6ebf063d7 <unknown>
#11 0x55a6ebf10b20 <unknown>
#12 0x55a6ebf07023 <unknown>
#13 0x55a6ebed51aa <unknown>
#14 0x55a6ebf2b6b8 <unknown>
#15 0x55a6ebf2b847 <unknown>
#16 0x55a6ebf3b243 <unknown>
#17 0x7ffb30c27609 start_thread

how to work around!? Well - i am trying to figure out the simplest approach to gather data from clutch.io

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
malaga
  • 319
  • 2
  • 13
  • 3
    You need to pass the path to your chromedriver – Celius Stingher Jun 05 '23 at 18:28
  • 1
    dear Celius Stingher - i love to hear from you again . – malaga Jun 07 '23 at 09:10
  • 3
    Hi @malaga, was undetected Selenium's answer not helpful? Thanksfor adding that you are running this in a google-colab environment. Is there something else that should be taken into account before I try to provide an answer? – Celius Stingher Jun 07 '23 at 10:21
  • 2
    hello dear Celius: many thanks for your reply. No afaik nothing helped so far. i am wondering why i cannot work out this on Colab: see besides this approach all the others: a. https://stackoverflow.com/questions/76401453/ b. https://stackoverflow.com/questions/76401129/ c. https://stackoverflow.com/questions/76389587/ d. https://stackoverflow.com/questions/76362600/ Look forward to hear from you. BTW i am workin on a Anaconda-setup here - but this takes some time - if colab-approaches fail all the time - i try to work it out on anaconda. Love to hear from you. Regards – malaga Jun 07 '23 at 13:17
  • 1
    hi there dear @CeliusStingher - do you have any ideas to go ahead here!? – malaga Jun 19 '23 at 09:14
  • @Celius Stingher i would love to hear from you again - have a great day. – malaga Jun 20 '23 at 15:13

1 Answers1

2

This error message...

WebDriverException: Message: unknown error: cannot find Chrome binary

...implies that the ChromeDriver was unable to find the binary in the default location of your system or it wasn't installed at all.

Using a Chrome executable in a non-standard location

You need to override the default Chrome binary location with the custom location as follows:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.binary_location = "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe" # as per my Win10 OS box
s = Service('C:\\BrowserDrivers\\chromedriver.exe')
driver = webdriver.Chrome(service=s, options=options)
driver.get('http://google.com/')
print("Chrome Browser Invoked")

References

You can find a couple of relevant detailed discussions in:

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • hello dear undetected Selenium - many thanks - i am currently trying to get this working in my google-colab environment - so that it works on my goal - to gather data from clutch... – malaga Jun 06 '23 at 09:51