1

I get an error when trying to open a link via Selenium in:

  • FireFox V. 73. 0 (64-bit)
  • Geckodriver driver version-v0.26. 0

Help me solve the problem

public By searchResultLinktLocator(int searchResult) {
    By searchResultLinktLocator = By.xpath("//li[@data-result-number='" + searchResult + "']" + "//a");
    return searchResultLinktLocator;
}

public BBCSearchPage openSearchResultLink(int searchResult) {
    actions.click(driver.findElement(searchResultLinktLocator(searchResult))).perform();
    return this; 
}

The link for this function does not open: 

searchPage.openSearchResultLink(1);

A page that does not open a link: https://www.bbc.co.uk/search?q=Andersen

1581606071797   mozrunner::runner   INFO    Running command: "C:\\Program Files\\Mozilla Firefox\\firefox.exe" "-marionette" "-foreground" "-no-remote" "-profile" "C:\\Users\\anduser\\AppData\\Local\\Temp\\rust_mozprofileZpIGZz"
1581606072297   addons.webextension.doh-rollout@mozilla.org WARN    Loading extension 'doh-rollout@mozilla.org': Reading manifest: Invalid extension permission: networkStatus
1581606072791   addons.webextension.screenshots@mozilla.org WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: mozillaAddons
1581606072791   addons.webextension.screenshots@mozilla.org WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: telemetry
1581606072791   addons.webextension.screenshots@mozilla.org WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: resource://pdf.js/
1581606072791   addons.webextension.screenshots@mozilla.org WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: about:reader*
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
1581606075330   Marionette  INFO    Listening on port 53959
1581606075570   Marionette  WARN    TLS certificate errors will be ignored for this session
фев 13, 2020 6:01:15 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
JavaScript error: https://nav.files.bbci.co.uk/orbit/1b2e292884201dd13064a9204e177864/js/require.min.js, line 1: TypeError: e.slice is not a function
1581606081548   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
action.dispatch@chrome://marionette/content/action.js:1034:5
performActions@chrome://marionette/content/listener.js:850:16
dispatch/</req<@chrome://marionette/content/listener.js:527:14
dispatch/<@chrome://marionette/content/listener.js:520:15
1581606081697   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081712   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081730   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081732   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081734   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081747   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081752   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081753   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081755   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081755   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081863   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081874   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081882   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081886   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081888   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081899   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081931   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
1581606081945   Marionette  WARN    TimedPromise timed out after 0 ms: stacktrace:
TimedPromise/<@chrome://marionette/content/sync.js:244:13
TimedPromise@chrome://marionette/content/sync.js:229:10
Sleep@chrome://marionette/content/sync.js:282:10
dispatchPause@chrome://marionette/content/action.js:1482:10
toEvents/<@chrome://marionette/content/action.js:1156:16
action.dispatchTickActions@chrome://marionette/content/action.js:1060:35
action.dispatch/chainEvents<@chrome://marionette/content/action.js:1028:20
JavaScript error: , line 0: uncaught exception: undefined
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
Alexis_Shef_777
  • 109
  • 1
  • 1
  • 10

2 Answers2

1

I took your code and simplified the structure and ran the test with minimal lines of code as follows:

  • Code Block:

    public class A_Firefox_Test 
{
    public static void main(String[] args) 
    {
        System.setProperty("webdriver.gecko.driver", "C:/Utility/BrowserDrivers/geckodriver.exe");
        WebDriver driver = new FirefoxDriver();
        driver.get("https://www.bbc.co.uk/search?q=Andersen");
        int searchResult = 1;
        new WebDriverWait(driver, 10).until(ExpectedConditions.elementToBeClickable(By.xpath("//li[@data-result-number='" + searchResult + "']"))).click();
    }
}

Similar to your observation I have hit the same roadblock, that the desired link wasn't clickable and additionally an WARNING as:

[Parent 7472, Gecko_IOThread] WARNING: file z:/task_1579290903/build/src/ipc/chromium/src/base/process_util_win.cc, line 160

which implies the service was interupted within process_util_win.cc i.e. process module.

Deep dive

I went ahead and while inspecting the DOM Tree of the webpage it was observed that some of the <script> tag refers to JavaScripts having keyword dist. As an example:

  • map['idcta-v2/idcta-1'] = 'https://static.files.bbci.co.uk/account/id-cta/1.36.1/modules/idcta-v2/dist/idcta-1.min';
  • document.write('<script type="text/javascript" src="https://static.bbc.co.uk/bbcdotcom/3.6.969/script/dist/bbcdotcom.dev.js">\x3C/script>');
  • document.write('<script type="text/javascript" src="https://static.bbc.co.uk/bbcdotcom/3.6.969/script/dist/bbcdotcom.js">\x3C/script>');

Which is a clear indication that the website is protected by Bot Management service provider Distil Networks and the navigation by ChromeDriver gets detected and subsequently blocked.

Distil

As per the article There Really Is Something About Distil.it...:

Distil protects sites against automatic content scraping bots by observing site behavior and identifying patterns peculiar to scrapers. When Distil identifies a malicious bot on one site, it creates a blacklisted behavioral profile that is deployed to all its customers. Something like a bot firewall, Distil detects patterns and reacts.

Further,

"One pattern with Selenium was automating the theft of Web content", Distil CEO Rami Essaid said in an interview last week. "Even though they can create new bots, we figured out a way to identify Selenium the a tool they're using, so we're blocking Selenium no matter how many times they iterate on that bot. We're doing that now with Python and a lot of different technologies. Once we see a pattern emerge from one type of bot, then we work to reverse engineer the technology they use and identify it as malicious".

Reference

You can find a couple of relevant detailed discussions in:

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
0

You may be missing .build() before the .perform().

actions.click(driver.findElement(searchResultLinktLocator(searchResult))).build().perform()

Is this only failing on some searches, but working on others? You also didn't include where you're instantiating actions, that may help shed some light on the problem. If you update your post, I will update my answer.

scilence
  • 1,069
  • 1
  • 5
  • 10