Python Selenium Headless Error: Element not Interactable

Question

I was making a program which scrapes info about stocks from the website https://www.tradingview.com/screener/ which used selenium.here is the code:

def stock(aaa):
    a=[]
    chrome_options = Options()
    custom_header = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36'}
    chrome_options.add_argument("--headless")
    chrome_options.add_argument(f"user-agent={custom_header}")
    chrome_options.add_argument("--disable-gpu")
   
    browser = webdriver.Chrome(options=chrome_options)
    browser.get("https://www.tradingview.com/screener/")
    entry=browser.find_element_by_css_selector("#js-screener-container > div.tv-data-table-sticky-wrapper.tv-screener-sticky-header-wrapper > table > thead > tr > th.tv-data-table__th.tv-data-table__cell.js-tv-data-table-th.js-tv-data-table-th-name.tv-data-table__sortable.tv-screener-table__sortable.tv-screener-table__th.js-draggable > div > div > div.tv-screener-table__search-query.js-search-query.tv-screener-table__search-query--without-description>input")
    entry.send_keys(aaa)

It was working well when it was not in headless mode,but when I switched to headless,it gave this error code:

File "c:\Users\██████\cowination.py", line 181, in <module>
    aaa=stock("zydus")
  File "c:\Users\██████\cowination.py", line 20, in stock
    entry.send_keys(aaa)
  File "C:\Users\████████████\Python39\lib\site-packages\selenium\webdriver\remote\webelement.py", line 477, in send_keys
    self._execute(Command.SEND_KEYS_TO_ELEMENT,
  File "C:\Users\████████████\Python39\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "C:\Users\████████████\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Users\████████████\Python39\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
  (Session info: headless chrome=90.0.4430.212)

At first I thought it was because the window was not maximized as the entry box is not visible when not maximized.So I added the --start-maximized argument,but it too didn't work.I tried even Webdriverwait,but that to didn't work. So is there any way I can solve this problem? thanks in advance

Edit: I have written my code according to the correct answer.here is the code: from selenium import webdriver

from selenium.webdriver.chrome.options import Options
import time
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
def stock(aaa):
    a=[]
    chrome_options = Options()
    custom_header = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36'}
    chrome_options.add_argument("--headless")
    chrome_options.add_argument(f"user-agent={custom_header}")
    chrome_options.add_argument("--disable-gpu")
   
    browser = webdriver.Chrome(options=chrome_options)
    browser.get("https://www.tradingview.com/screener/")
    browser.maximize_window()
    time.sleep(3)
    e=browser.find_element_by_xpath("/html/body/div[6]/div/div[2]/div[8]/div[1]/img")
    e.click()
    from selenium.webdriver.common.action_chains import ActionChains
    ActionChains(browser).move_to_element(browser.find_element_by_css_selector('#js-screener-container > div.tv-screener-toolbar.tv-screener-toolbar--standalone.tv-screener-toolbar--standalone_sticky > div.tv-screener-market-select.tv-dropdown.tv-dropdown--inline.tv-dropdown-behavior.i-opened > div.tv-dropdown__body.tv-dropdown-behavior__body.tv-dropdown__body--over-trigger.tv-dropdown__body--position_right.undefined.i-opened > div > div.tv-dropdown-behavior__inscroll.js-content > div:nth-child(19) > div > span')).perform()
    india=browser.find_element_by_css_selector('#js-screener-container > div.tv-screener-toolbar.tv-screener-toolbar--standalone.tv-screener-toolbar--standalone_sticky > div.tv-screener-market-select.tv-dropdown.tv-dropdown--inline.tv-dropdown-behavior.i-opened > div.tv-dropdown__body.tv-dropdown-behavior__body.tv-dropdown__body--over-trigger.tv-dropdown__body--position_right.undefined.i-opened > div > div.tv-dropdown-behavior__inscroll.js-content > div:nth-child(19) > div > span')
    india.click()
    time.sleep(1)
    filters=browser.find_element_by_css_selector("#js-screener-container > div.tv-screener-toolbar.tv-screener-toolbar--standalone.tv-screener-toolbar--standalone_sticky > div.tv-screener-toolbar__button.tv-screener-toolbar__button--options.tv-screener-toolbar__button--filters.apply-common-tooltip.common-tooltip-fixed")
    filters.click()
    time.sleep(1)
    reset=browser.find_element_by_css_selector("#overlap-manager-root > div > div > div > div.tv-screener-search.js-screener-search > div")
    reset.click()
    exitbut=browser.find_element_by_css_selector("#overlap-manager-root > div > div > div > div.tv-dialog__section.tv-dialog__section--title.tv-tabbed-dialog__header.js-close-button-place > div.tv-dialog__close.js-dialog__close.tv-tabbed-dialog__close")
    exitbut.click()
    time.sleep(3)
    entry=browser.find_element_by_css_selector("#js-screener-container > div.tv-data-table-sticky-wrapper.tv-screener-sticky-header-wrapper > table > thead > tr > th.tv-data-table__th.tv-data-table__cell.js-tv-data-table-th.js-tv-data-table-th-name.tv-data-table__sortable.tv-screener-table__sortable.tv-screener-table__th.js-draggable > div > div > div.tv-screener-table__search-query.js-search-query.tv-screener-table__search-query--without-description>input")
    
    browser.execute_script("$('.tv-screener-table__search-input.js-search-input').val('{}');".format(aaa))
    action = webdriver.ActionChains(browser);
    action.send_keys(Keys.RETURN).send_keys(Keys.BACKSPACE).perform()
    time.sleep(6)
    p=browser.find_element_by_css_selector("#js-screener-container > div.tv-screener__content-pane > table > tbody > tr:nth-child(1) > td.tv-data-table__cell.apply-common-tooltip.tv-screener-table__cell.tv-screener-table__cell--left.tv-screener-table__cell--with-marker > div > div:nth-child(3) > span.tv-screener__description")
    p1=browser.find_element_by_css_selector("#js-screener-container > div.tv-screener__content-pane > table > tbody > tr:nth-child(1) > td:nth-child(2) > span")
    a.append(p.text)
    a.append(p1.text)
    p2=browser.find_element_by_css_selector("#js-screener-container > div.tv-screener__content-pane > table > tbody > tr:nth-child(1) > td:nth-child(3)")
    p3=browser.find_element_by_css_selector("#js-screener-container > div.tv-screener__content-pane > table > tbody > tr:nth-child(1) > td:nth-child(4)")
    a.append(p2.text)
    a.append(p3.text)
    p4=browser.find_element_by_css_selector("#js-screener-container > div.tv-screener__content-pane > table > tbody > tr:nth-child(1) > td:nth-child(5) > span")
    a.append(p4.text)
    p5=browser.find_element_by_css_selector("#js-screener-container > div.tv-screener__content-pane > table > tbody > tr:nth-child(1) > td:nth-child(6)")
    a.append(p5.text)
    p6=browser.find_element_by_css_selector("#js-screener-container > div.tv-screener__content-pane > table > tbody > tr:nth-child(1) > td:nth-child(7)")
    a.append(p6.text)
    p7=browser.find_element_by_css_selector("#js-screener-container > div.tv-screener__content-pane > table > tbody > tr:nth-child(1) > td:nth-child(8)")
    a.append(p7.text)
    p8=browser.find_element_by_css_selector("#js-screener-container > div.tv-screener__content-pane > table > tbody > tr:nth-child(1) > td:nth-child(9)")
    a.append(p8.text)

    p10=browser.find_element_by_css_selector("#js-screener-container > div.tv-screener__content-pane > table > tbody > tr:nth-child(1) > td:nth-child(11)")
    a.append(p10.text)
    return f"""Name of stock:{a[0]}
    Last price:{a[1]}
    Change %:{a[2]}
    Change value:{a[3]}
    Rating:{a[4]}
    Volume:{a[5]}
    Market Cap:{a[6]}
    Price to Earning Ratio:{a[7]}
    EPS(TTM):{a[8]}
    Sector:{a[9]}
    """

here is the output :

Name of stock:RELIANCE INDUSTRIES LTD.
    Last price:2000.90
    Change %:0.79%
    Change value:15.60
    Rating:Buy
    Volume:210.603K
    Market Cap:12.656T
    Price to Earning Ratio:—
    EPS(TTM):—
    Sector:Energy Minerals

this is the expected output:

Name of stock:ZYDUS WELLNESS LTD.
        Last price:2110.15
        Change %:-0.57%
        Change value:-12.15
        Rating:Buy
        Volume:24.03K
        Market Cap:134.251B
        Price to Earning Ratio:114.58
        EPS(TTM):18.52
        Sector:Consumer Non-Durables

Now it is typing in the search box but giving wrong output which is actually the info of the stock at the homepage and not that of the search results . Can anyone tell why it is happening?

Actually it would return some info about the stock that was searched but I gave only the part of the code that was raising the error as the other code was very lenghty — Code2D, May 21 '21 at 00:54

Tony · Accepted Answer · 2021-05-21T05:54:22.977

0

Because you are using headless mode the send_keys won't work in this case. See also this post. When you remove headless option and replace the "zydus" with e.g. "aapl" everything works fine.

One ways to work this around in your specific case would be e.g.:

from selenium.webdriver.common.keys import Keys

browser.execute_script("$('.tv-screener-table__search-input.js-search-input').val('{}');".format(aaa))
action = webdriver.ActionChains(browser);
action.send_keys(Keys.SPACE).send_keys(Keys.BACKSPACE).perform()

This works exactly the same way as entry.send_keys(aaa)

edited May 21 '21 at 05:54

answered May 20 '21 at 23:30

Tony

7,767
2
22
51

So is there any alternative for send_keys? – Code2D May 21 '21 at 00:40
now the thing is it is entering the search into the input but it seems to not return the results – Code2D May 21 '21 at 11:05
This is a different question which does't fit anymore into the original post. To ask a new question make a new post: one question per post. – Tony May 21 '21 at 12:18

Python Selenium Headless Error: Element not Interactable

1 Answers1