Amazon Ubuntu EC2 Python Selenium Webbrowser Chrome Element suddenly not interactable anymore

Question

I successfully created a python script that downloads stock data from the charting platform tradingview. When i start it from my local machine everything works fine.

Now i wanted to start the script from an ubuntu ec2 machine and a crontab. I first tried to run the script manually from the ec2 machine a couple days ago. Today i wanted to try to set up the crontab. It didnt work, so i tried to start the python script manually again. This time I get this error.

File "automated_prediction_ec2.py", line 282, in <module>
    signin.click()
  File "/home/ubuntu/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
  (Session info: headless chrome=86.0.4240.75)

First I thought it could be that tradingview changed their HTML, so I tried to run it from my local machine. This worked perfectly as before.

These are my ChromeOptions. I already tried several combinations to solve the problem but these are the ones where it worked last time.

options = webdriver.ChromeOptions()
prefs = {'download.default_directory' : '/home/ubuntu/daily_stock_data'}
options.add_argument('--headless')
options.add_argument("start-maximized")
options.add_argument("disable-infobars")
options.add_argument("--disable-extensions")
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--no-sandbox')
options.add_experimental_option('prefs', prefs)

So what could be the problem, that one day my script is working from the ec2 machine and then suddenly i get this error? Furthermore the script is downloading 130 CSVs. Is there something like a download limit for a ec2 instance?

Here is the code where the error occurs.

driver.get("https://www.tradingview.com/")
driver.implicitly_wait(20)
signin = driver.find_element_by_class_name("tv-header__link.tv-header__link--signin.js-header__signin")
signin.click()

HTML Snippet of the page.

<span class="tv-header__dropdown-text">
<a class="tv-header__link tv-header__link--signin js-header__signin" href="#signin">Sign in</a>
</span>

This is not a chrome initialization problem. It's a race condition between selenium and your script, because JavaScript on the web page is doing something to modify the page at runtime without selenium waiting for it. Please add the python code in your script, and a relevant snippet of the HTML on the page. — Greg Burghardt, Nov 11 '20 at 17:50

undetected Selenium · Answer 1 · 2020-11-11T22:19:48.093

To click on the element with text as Sign in you can use either of the following Locator Strategies:

Using link_text:

driver.find_element_by_link_text("Sign in").click()

Using css_selector:

driver.find_element_by_css_selector("a.tv-header__link--signin[href='#signin']").click()

Using xpath:

driver.find_element_by_xpath("//a[contains(@class, 'tv-header__link--signin') and text()='Sign in']").click()

Ideally, to click on the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:

Using LINK_TEXT:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, "Sign in"))).click()

Using CSS_SELECTOR:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.tv-header__link--signin[href='#signin']"))).click()

Using XPATH:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[contains(@class, 'tv-header__link--signin') and text()='Sign in']"))).click()

Note: You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

I tried the WebDriverWait and the Locator Strategies. Got an TimeOut Error at another point. Tried to solve it with ```presence_of_element_located``` but now i get this error ```driver = webdriver.Chrome(options=options) ... selenium.common.exceptions.WebDriverException: Message: unknown error: DevToolsActivePort file doesn't exist``` — Russgo, Nov 16 '20 at 13:28
@Russgo Let's discuss in details within [Selenium Chat Room](https://chat.stackoverflow.com/rooms/223360/selenium) — undetected Selenium, Nov 16 '20 at 14:09

Amazon Ubuntu EC2 Python Selenium Webbrowser Chrome Element suddenly not interactable anymore

1 Answers1