I want to webscraping with python in a host without Gui and because this webpage (https://aqms.doe.ir/App/) don't access to scrap with requests library, I have to do it by selenium in headless webdriver. (Sorry! My English is bad!)
so I run this code in a cpanel host by terminal:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from time import sleep
from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
with Chrome(options=chrome_options) as driver:
driver.get("https://aqms.doe.ir/App/")
sleep(10)
refresh = driver.find_element(By.XPATH, '/html/body/app-root/app-toolbar/div/div/app-toolbar-port/mat-toolbar/mat-toolbar-row[1]/button[2]/span/mat-icon')
refresh.click()
sleep(10)
shn = driver.find_element(By.CSS_SELECTOR, '#highcharts-29 > div > div:nth-child(1) > span > div > span:nth-child(1)').text
print (shn)
NOETS:
I have installed selenium in the host with terminal. and the chromewebdriver.exe is in the folder that code is there (the source and chromewebdriver are in one folder). I run this code with creating an application in cpanel by 'setup python' and runed it in the terminal.
but I got this Error:
File "req2.py", line 10, in <module>
with Chrome(options=chrome_options) as driver:
File "/home/gigachad/virtualenv/python_bot/3.8/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
super().__init__(
File "/home/gigachad/virtualenv/python_bot/3.8/lib/python3.8/site-packages/selenium/webdriver/chromium/webdriver.py", line 103, in __init__
self.service.start()
File "/home/gigachad/virtualenv/python_bot/3.8/lib/python3.8/site-packages/selenium/webdriver/common/service.py", line 106, in start
self.assert_process_still_running()
File "/home/gigachad/virtualenv/python_bot/3.8/lib/python3.8/site-packages/selenium/webdriver/common/service.py", line 117, in assert_process_still_running
return_code = self.process.poll()
AttributeError: 'Service' object has no attribute 'process'
I think it's because of running it in the host.
thanks a lot!