3

I am using Selenium 3.141.0 with Python, for Firefox 88.0 with geckodriver 0.29.1.

Previously, I was able to run the below JavaScript execution to remove unwanted characters (in this case ®) from web pages, as demonstrated in this answer:

driver.execute_script("var replaced = $('body').html().replace(/(®)/g,''); $('body').html(replaced);")

However, at some point recently this no longer works, giving the following error:

JavascriptException: Message: SecurityError: The operation is insecure.

Presumably, when updating Selenium, Firefox, or geckodriver, this function broke – unless there is something on the web page that I crawl (which is not mine) that has changed, that is making the script impossible to execute (I don't know what on the web page possibly would cause this).

What could be the reason behind this issue, and is it possible to "override" the SecurityError and execute the script?

P A N
  • 5,642
  • 15
  • 52
  • 103

3 Answers3

1

Have you tried anything like this?

  1. Go to about:Config and verify that dom.serviceWorkers.enabled is set to true
  2. Then, go to about:preferences#privacy and ensure that you have unchecked Delete cookies and site data when Firefox is closed
C. Peck
  • 3,641
  • 3
  • 19
  • 36
1

Seems like the script is trying to update/replace the content - $('body').html(replaced); Maybe after updating it is trying to download some content due to which the error SecurityError: The operation is insecure is thrown.

To triangulate/debug this issue, you may breakdown the script execution and check on which statement the exception occurs -

  • driver.execute_script("var replaced = $('body').html().replace(/(®)/g,'');")
  • driver.execute_script("$('body').html(replaced);")

Also, this behaviour does not seem to be with Selenium/GeckoDriver itself. But with the Javascript/JQuery code being executed.

I found these two StackOverflow questions around JQuery + above error which might point you in the right direction -

  • Link 1
  • Link 2
  • Link 3 (this is as mentioned in the above comment) Here you may try to set pref via Selenium Capabilities to set the values for dom.serviceWorkers.enabled

Will update this answer if I found some other way to handle this behaviour

Xtraterrestrial
  • 651
  • 1
  • 6
  • 12
1

Thanks to @Xtraterrestrial for providing links in their answer that helped me solve the issue. I have awarded my bounty.

For clarity, I am here providing my particular solution.

As described in one of the linked answers, the SecurityError arises apparently because the JavaScript attempts to modify insecure elements on the web page, such as <input>.

Using the code from this answer instead of the original JQuery used, the script now looks for TextNodes and modifies only those.

var replaceTextInNode = function(parentNode){
    for(var i = parentNode.childNodes.length-1; i >= 0; i--){
        var node = parentNode.childNodes[i];

        //  Make sure this is a text node
        if(node.nodeType == Element.TEXT_NODE){
            node.textContent = node.textContent.replace(/(®)/g,'')
        } else if(node.nodeType == Element.ELEMENT_NODE){
            //  Check this node's child nodes for text nodes to act on
            replaceTextInNode(node);
        }
    }
};

replaceTextInNode(document.body);

Implemented in Python for Selenium as:

driver.execute_script("var replaceTextInNode = function(parentNode){for(var i = parentNode.childNodes.length-1; i >= 0; i--){var node = parentNode.childNodes[i]; if(node.nodeType == Element.TEXT_NODE){node.textContent = node.textContent.replace(/(®)/g,'')} else if(node.nodeType == Element.ELEMENT_NODE){replaceTextInNode(node); }}}; replaceTextInNode(document.body);")
P A N
  • 5,642
  • 15
  • 52
  • 103