1

I'm trying to web scrape using selenium an NFT, website Magic Eden. When I try and scrape normal data like text on top of the page my code works fine, but when I try to select an NFT category I get the message

Message: invalid selector: An invalid or illegal selector was specified

I also directly copied the selector so I know it isn't a typo.

from selenium import webdriver
from selenium.webdriver import Chrome
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()

driver.get("https://magiceden.io/collections?type=popular")

content = driver.find_element(By.CSS_SELECTOR,"#root > div > div > div > div:nth-child(3) > div.tw-relative.tw-flex.tw-flex-col.tw-flex-auto.tw-ml-0.content__regular.lg\:tw-ml-240px > section > div.infinite-scroll-component__outerdiv > div > div.tw-grid.tw-gap-3.tw-grid-cols-2.md\:tw-grid-cols-3.lg\:tw-grid-cols-4.xl\:tw-grid-cols-4.\33 xl\:tw-grid-cols-4").click()
 # example of css selection that gives me an error

driver.close()

full error:

DevTools listening on ws://127.0.0.1:61903/devtools/browser/4053cbe0-4a71-412a-88cd-ee0237b0a9a4
Traceback (most recent call last):
  File "C:\Users\colli\Desktop\Solana\main.py", line 11, in <module>
    content = driver.find_element(By.CSS_SELECTOR,"#root > div > div > div > div:nth-child(3) > div.tw-relative.tw-flex.tw-flex-col.tw-flex-auto.tw-ml-0.content__regular.lg\:tw-ml-240px > section > div.infinite-scroll-component__outerdiv > div > div.tw-grid.tw-gap-3.tw-grid-cols-2.md\:tw-grid-cols-3.lg\:tw-grid-cols-4.xl\:tw-grid-cols-4.\33 xl\:tw-grid-cols-4").click()
  File "C:\Users\colli\Desktop\Solana\.venvName\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1248, in find_element
    return self.execute(Command.FIND_ELEMENT, {
  File "C:\Users\colli\Desktop\Solana\.venvName\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 425, in execute
    self.error_handler.check_response(response)
  File "C:\Users\colli\Desktop\Solana\.venvName\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified
  (Session info: chrome=98.0.4758.102)
Stacktrace:
Backtrace:
        Ordinal0 [0x00BC69A3+2582947]
        Ordinal0 [0x00B5A6D1+2139857]
        Ordinal0 [0x00A53A98+1063576]
        Ordinal0 [0x00A562B7+1073847]
        Ordinal0 [0x00A5617E+1073534]
        Ordinal0 [0x00A563F0+1074160]
        Ordinal0 [0x00A7FCB2+1244338]
        Ordinal0 [0x00A8013B+1245499]
        Ordinal0 [0x00AA9F52+1417042]
        Ordinal0 [0x00A98594+1344916]
        Ordinal0 [0x00AA834A+1409866]
        Ordinal0 [0x00A98366+1344358]
        Ordinal0 [0x00A75176+1200502]
        Ordinal0 [0x00A76066+1204326]
        GetHandleVerifier [0x00D6BE02+1675858]
        GetHandleVerifier [0x00E2036C+2414524]
        GetHandleVerifier [0x00C5BB01+560977]
        GetHandleVerifier [0x00C5A8D3+556323]
        Ordinal0 [0x00B6020E+2163214]
        Ordinal0 [0x00B65078+2183288]
        Ordinal0 [0x00B651C0+2183616]
        Ordinal0 [0x00B6EE1C+2223644]
        BaseThreadInitThunk [0x7694FA29+25]
        RtlGetAppContainerNamedObjectPath [0x77B37A9E+286]
        RtlGetAppContainerNamedObjectPath [0x77B37A6E+238]


(.venvName) C:\Users\colli\Desktop\Solana>[24804:9788:0302/175300.451:ERROR:chrome_browser_main_extra_parts_metrics.cc(227)] START: ReportBluetoothAvailability(). If you don't see the END: message, this is crbug.com/1216328.
[24804:9788:0302/175300.451:ERROR:chrome_browser_main_extra_parts_metrics.cc(230)] END: ReportBluetoothAvailability()
[24804:9788:0302/175300.453:ERROR:chrome_browser_main_extra_parts_metrics.cc(235)] START: GetDefaultBrowser(). If you don't see the END: message, this is crbug.com/1216328.
[24804:20312:0302/175300.454:ERROR:device_event_log_impl.cc(214)] [17:53:00.454] USB: usb_device_handle_win.cc:1049 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)
[24804:20312:0302/175300.456:ERROR:device_event_log_impl.cc(214)] [17:53:00.456] USB: usb_device_handle_win.cc:1049 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)
[24804:9788:0302/175300.465:ERROR:chrome_browser_main_extra_parts_metrics.cc(239)] END: GetDefaultBrowser()
marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
SMITHIN COMEDY
  • 37
  • 6

1 Answers1

1

This error message...

selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified

...implies that the following wasn't a valid css_selector:

#root > div > div > div > div:nth-child(3) > div.tw-relative.tw-flex.tw-flex-col.tw-flex-auto.tw-ml-0.content__regular.lg\:tw-ml-240px > section > div.infinite-scroll-component__outerdiv > div > div.tw-grid.tw-gap-3.tw-grid-cols-2.md\:tw-grid-cols-3.lg\:tw-grid-cols-4.xl\:tw-grid-cols-4.\33 xl\:tw-grid-cols-4

It is to be noted that the : character has a special meaning when used within a css_selector. As an example:

  • div:first-child
  • div:first-of-type

Hence, : can't be used as:

  • .lg\:tw-ml-240px
  • .md\:tw-grid-cols-3, etc
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352