Selenium (via Python) errors on SOME (but not all) :not(...) CSS selectors

Question

Running Python 2.7.8 & Selenium 3.11.0 (with Chrome webdriver), seeing some weird behavior. tl/dr, trying to use find_elements_by_css_selector, and it IS respecting a selector like p:not(.ignore) but NOT p:not(.ignore p) (though they both work in the real browser.

I'm serving the following webpage for test purposes:

<html>
    <title>Test</title>
    <body>
        <section class="ignore">
            <p>Some content I don't want to pull.</p>
        </section>
        <p>Content I DO want to pull.</p>
        <p>More important content.</p>
        <p>Thanks for reading.</p>
        <p class="also-ignore">(We CAN successfully ignore this one tho.)</p>
    </body>
</html>

And accessing it via Selenium with the following script:

#! /usr/bin/env python

from selenium import webdriver

if __name__ == '__main__':
    driver = webdriver.Chrome()

    try:
        driver.get('http://localhost:8000/test.html')
        elems = driver.find_elements_by_css_selector('p:not(.ignore p)')
        for e in elems:
            print e.text
    finally:
        driver.close()

The above throws the error:

Traceback (most recent call last):
  File "./test.py", line 10, in <module>
    elems = driver.find_elements_by_css_selector('p:not(.ignore p)')
  File "/Users/maiamccormick/code/seleniumtest/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 605, in find_elements_by_css_selector
    return self.find_elements(by=By.CSS_SELECTOR, value=css_selector)
  File "/Users/maiamccormick/code/seleniumtest/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 983, in find_elements
    'value': value})['value'] or []
  File "/Users/maiamccormick/code/seleniumtest/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 312, in execute
    self.error_handler.check_response(response)
  File "/Users/maiamccormick/code/seleniumtest/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified
  (Session info: chrome=65.0.3325.181)
  (Driver info: chromedriver=2.37.544337 (8c0344a12e552148c185f7d5117db1f28d6c9e85),platform=Mac OS X 10.13.4 x86_64)

(The selector p:not(.ignore p) DOES work in the Chrome console.)

The above Python code runs without error (and with expected output) with the CSS selector p:not(.also-ignore) (i.e. ignoring the last <p> element in the page), so the problem does not seem to be with the :not(...) selector itself. I'm stumped -- anyone have thoughts?

FWIW, replicated this behavior with the Firefox webdriver as well (same selectors do/don't work using Firefox webdriver -- and all work in Firefox console). — maiamcc, Apr 09 '18 at 07:46
Apart from what is working and what not do you have a question for us? What is your exact usecase? — undetected Selenium, Apr 09 '18 at 07:58
@DebanjanB tbh I'm not actually looking for help making something WORK -- I am easily able to hack around the problem above. I'm more curious WHY the selector didn't work as I expected it to. — maiamcc, Apr 12 '18 at 17:17

Florent B. · Accepted Answer · 2018-04-09T15:05:23.810

From the W3C specification:

The negation pseudo-class, :not(X), is a functional notation taking a simple selector (excluding the negation pseudo-class itself) as an argument. It represents an element that is not represented by its argument.

This means that p:not(.ignore p) is not supported.

Now the reason it's working in the console is because the shorthand $ is overwritten by the page with JQuery which supports a different syntax.

Since Selenium rely on the CSS selector from the browser (same as document.querySelector) and not from JQuery, the exception in this case is to be expected.

Note that you'll get the exception in the console by either executing document.querySelector('p:not(.ignore p)') or $('p:not(.ignore p)') in a blank page (about:blank).

If you wish to use a JQuery selector, then use a JavaScript injection:

elms = driver.execute_script("return $('p:not(.ignore p)')")

See also: [Why is my jQuery :not() selector not working in CSS?](https://stackoverflow.com/questions/10711730/why-is-my-jquery-not-selector-not-working-in-css) — BoltClock, Apr 09 '18 at 17:40

Selenium (via Python) errors on SOME (but not all) :not(...) CSS selectors

1 Answers1