0

I want to webscraping with python in a host without Gui and because this webpage (https://aqms.doe.ir/App/) don't access to scrap with requests library, I have to do it by selenium in headless webdriver. (Sorry! My English is bad!)

so I run this code in a cpanel host by terminal:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from time import sleep
from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options

chrome_options = Options()  
chrome_options.add_argument("--headless")
with Chrome(options=chrome_options) as driver:
    driver.get("https://aqms.doe.ir/App/")
    sleep(10)
    refresh = driver.find_element(By.XPATH, '/html/body/app-root/app-toolbar/div/div/app-toolbar-port/mat-toolbar/mat-toolbar-row[1]/button[2]/span/mat-icon')
    refresh.click()
    sleep(10)
    shn = driver.find_element(By.CSS_SELECTOR, '#highcharts-29 > div > div:nth-child(1) > span > div > span:nth-child(1)').text
    print (shn)


NOETS:

I have installed selenium in the host with terminal. and the chromewebdriver.exe is in the folder that code is there (the source and chromewebdriver are in one folder). I run this code with creating an application in cpanel by 'setup python' and runed it in the terminal.

but I got this Error:

  File "req2.py", line 10, in <module>
    with Chrome(options=chrome_options) as driver:
  File "/home/gigachad/virtualenv/python_bot/3.8/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
    super().__init__(
  File "/home/gigachad/virtualenv/python_bot/3.8/lib/python3.8/site-packages/selenium/webdriver/chromium/webdriver.py", line 103, in __init__
    self.service.start()
  File "/home/gigachad/virtualenv/python_bot/3.8/lib/python3.8/site-packages/selenium/webdriver/common/service.py", line 106, in start
    self.assert_process_still_running()
  File "/home/gigachad/virtualenv/python_bot/3.8/lib/python3.8/site-packages/selenium/webdriver/common/service.py", line 117, in assert_process_still_running
    return_code = self.process.poll()
AttributeError: 'Service' object has no attribute 'process'

I think it's because of running it in the host.

thanks a lot!

1 Answers1

0

You need to pass a Service object as an argument to Chrome() pointing to the installation location of the chromedriver binary object as follows:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from time import sleep
from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager

chrome_options = Options()  
chrome_options.add_argument("--headless")
with Chrome(service=Service(ChromeDriverManager().install()), options=chrome_options) as driver:
    driver.get("https://aqms.doe.ir/App/")
    sleep(10)
    refresh = driver.find_element(By.XPATH, '/html/body/app-root/app-toolbar/div/div/app-toolbar-port/mat-toolbar/mat-toolbar-row[1]/button[2]/span/mat-icon')
    refresh.click()
    sleep(10)
    shn = driver.find_element(By.CSS_SELECTOR, '#highcharts-29 > div > div:nth-child(1) > span > div > span:nth-child(1)').text
    print (shn)

References

You can find a couple of relevant detailed discussions in:

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352