Selenium java code runs test successfully but fails when run as headless

Question

My java selenium java test case with Chrome version 77 runs fine when options.setHeadless(false); but fails when I change this line options.setHeadless(true);

The java code was exported by the katalon selenium recorder chrome extension.

Output when it runs fine with options.setHeadless(false);:

Title of the page is 9 -> Scan History
Title of the page is 10 -> Scan History
Title of the page is 10.1 -> Scan History
Title of the page is 10.2 -> Scan History
Title of the page is 10.3 -> Scan History
Title of the page is 10.4 -> Scan History
Title of the page is 10.5 -> Scan History

When I run the same code with options.setHeadless(true); I get a below warning as well as an error message and the execution terminates before completion.

Title of the page is 10 -> Scan History
[0530/125542.102:INFO:CONSOLE(0)] "Error parsing header X-XSS-Protection: 1; mod
e=block, 1;mode=block: expected semicolon at character position 13. The default
protections will be applied.", source: https://qualysguard.myshop.com/fo/scan/sca
nList.php (0)
Title of the page is 10.1 -> Scan History
Title of the page is 10.2 -> Scan History
Title of the page is 10.3 -> Scan History
Exception in thread "main" org.openqa.selenium.NoSuchElementException: no such e
lement: Unable to locate element: {"method":"css selector","selector":"#ext\-gen
117"}
  (Session info: headless chrome=77.0.3865.75)
For documentation on this error, please visit: https://www.seleniumhq.org/except
ions/no_such_element.html
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17
:03'
System info: host: 'myhostserver', ip: '10.9.111.32', os.name: 'Windows Serve
r 2012 R2', os.arch: 'amd64', os.version: '6.3', java.version: '1.8.0_45'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 7
7.0.3865.75, chrome: {chromedriverVersion: 77.0.3865.40 (f484704e052e0..., userD
ataDir: C:\Users\1886\AppData\Loc...}, goog:chromeOptions: {debuggerAddress: l
ocalhost:56128}, javascriptEnabled: true, networkConnectionEnabled: false, pageL
oadStrategy: normal, platform: XP, platformName: XP, proxy: Proxy(), setWindowRe
ct: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 30
0000, script: 30000}, unhandledPromptBehavior: dismiss and notify}
Session ID: 09a309a747f7bcb42735bb8b6c39ad1f
*** Element info: {Using=id, value=ext-gen117}
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)

        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Sou
rce)
        at java.lang.reflect.Constructor.newInstance(Unknown Source)
        at org.openqa.selenium.remote.http.W3CHttpResponseCodec.createException(
W3CHttpResponseCodec.java:187)
        at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpRe
sponseCodec.java:122)
        at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpRe
sponseCodec.java:49)
        at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExe
cutor.java:158)
        at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(Driv
erCommandExecutor.java:83)
        at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.ja
va:552)
        at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDrive
r.java:323)
        at org.openqa.selenium.remote.RemoteWebDriver.findElementById(RemoteWebD
river.java:372)
        at org.openqa.selenium.By$ById.findElement(By.java:188)
        at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDrive
r.java:315)
        at pack.QualysScan.testQualysScan(QualysScan.java:182)
        at pack.QualysScan.main(QualysScan.java:284)

Here is the part of the code concerning the issue:

System.out.println("Title of the page is 10 -> " + driver.getTitle());
driver.findElement(By.id("body-header")).click();
driver.get("https://qualysguard.myshop.com/fo/scan/scanList.php");
System.out.println("Title of the page is 10.1 -> " + driver.getTitle());
driver.findElement(By.xpath("(.//*[normalize-space(text()) and normalize-space(.)='Add to my Calendar'])[1]/following::em[2]")).click();
System.out.println("Title of the page is 10.2 -> " + driver.getTitle());
System.out.println("Title of the page is 10.3 -> " + driver.getTitle());
driver.findElement(By.id("ext-gen117")).click();
System.out.println("Title of the page is 10.4 -> " + driver.getTitle());
System.out.println("Title of the page is 10.5 -> " + driver.getTitle());
String your_title = "Launch Compliance Scan";

And here is how the driver is setup:

public static void setUp() throws Exception {
          System.setProperty("webdriver.chrome.driver", "H:\\Downloads\\Qualys\\vdi\\chromedriver.exe");
          ChromeOptions options =  new ChromeOptions();
          options.setHeadless(true);
          DesiredCapabilities capabilities = DesiredCapabilities.chrome();
 capabilities.setCapability(CapabilityType.ForSeleniumServer.ENSURING_CLEAN_SESSION, true);
    capabilities.setCapability("chrome.switches", Arrays.asList("--incognito"));
    options.merge(capabilities);
    driver = new ChromeDriver(options);
    driver.manage().timeouts().implicitlyWait(60, TimeUnit.SECONDS);
  }

I'm sharing snapshot of element ext\-gen117 as visible on the portal.

Video of the issue:

https://www.youtube.com/watch?v=abSwsPtOOG4

Can you please suggest how can I overcome the issue?

Answer 1

This is likely caused by a race condition between the browser and selenium, where selenium attempts to click on something before it exists in the DOM. Using an explicit wait should resolve the issue:

WebDriverWait wait = new WebDriverWait (driver, 30);
WebElement element = wait.until(ExpectedConditions.elementToBeClickable(By.id("ext-gen117");
element.click();