1

It works well when done in non-headless mode but in headless mode no elements are detected.

code

url: https://moneyforward.com/users/sign_in

I wanna type into mail_address and password and click button in headless mode.

import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import chromedriver_binary

options = Options()
options.add_argument("--headless")
options.add_argument("--disable-gpu")
driver = webdriver.Chrome(options=options)

driver.get("https://moneyforward.com/users/sign_in")

time.sleep(5)


# type into mail_address
driver.find_element_by_id('sign_in_session_service_email').send_keys({mail_address}])

# type into passowrd
driver.find_element_by_id('sign_in_session_service_password').send_keys({password})

# click login button
driver.find_element_by_id("login-btn-sumit").click()

The error statement is as follows.

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="sign_in_session_service_email"]"}
  (Session info: headless chrome=80.0.3987.132)

Why doesn't it work in headless mode when it works in non-headless mode? Help me.

Naoki Oshiumi
  • 91
  • 1
  • 10

3 Answers3

0

Try to wait for the element rather than just using a hard coded sleep:

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC    

WebDriverWait(driver, 20).until(
                EC.presence_of_element_located((By.ID, "sign_in_session_service_email"))

Before you try that you could crank up the sleep to a higher wait, jsut to debug.

It's also possible that in headless mode your "screen size" isn't configured correctly and is causing a responsive reaction to the screen causing the elements to change.

DMart
  • 2,401
  • 1
  • 14
  • 19
0

It seems the page you are accessing does not allow headless browser mode. Try printing the HTML page and see if it returns all elements. I tried by adding the following statement and looks like the application returns the forbidden page.

print(driver.find_element_by_xpath('//*').get_attribute("innerHTML"))

I got this from the URL you have shared.

Forbidden

You can refer to Chrome headless options for debugging

Deepak Kumar
  • 106
  • 7
0

I took your code and modified a bit and executed at my end and here is the execution result:

  • Code Block:

    from selenium import webdriver

options = webdriver.ChromeOptions()
options.headless = True
options.add_argument('window-size=1400,600')
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)

driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://moneyforward.com/users/sign_in')
print(driver.page_source)
driver.save_screenshot('./save_screenshot_method.png') #Capture the screen
driver.quit()

  • Console Output:

    <html><head></head><body><pre style="word-wrap: break-word; white-space: pre-wrap;">Forbidden</pre></body></html>

  • Browser Snapshot:

save_screenshot_method.png

Analysis

It seems ChromeDriver driven is getting detected and being Access Denied the message Forbidden is shown.

Solution

As a solution you can adopt some strategies so ChromeDriver driven Chrome Browsing context doesn't gets detected and you can find a couple of detailed discussions in:

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352