1

I was making a webscraper to get gpu stocks from https://www.nvidia.com/en-us/shop/geforce/?page=1&limit=9&locale=en-us to get my hands on a 30 series card, I am using python with bs4 and selenium for this.

I want to load more shopping items, on the website it has this load more button. So I grabbed its class and made it so selenium clicks it:

driver.find_element_by_class_name("buy-link").click()

but it says that the element in non interactiable, HTML for the button

this

The exact error it gives me is:

Message: element click intercepted: Element <span class="extra_style buy-link" data-color="#76b900" data-secondary-color="#fff" style="visibility: visible; cursor: pointer;" data-mpn-code="NVGFT070">...</span> is not clickable at point (657, 594). Other element would receive the click: <div class="popBg" id="nv-buy-now-model" style="display: block;">...</div>

I don't know much HTML, how can I achieve clicking this button

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
Kryptic Coconut
  • 159
  • 1
  • 10

2 Answers2

1

The LOAD MORE element is a Angular element so to click on it you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:

  • Using CSS_SELECTOR:

    driver.get("https://www.nvidia.com/en-us/shop/geforce/?page=1&limit=9&locale=en-us")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div#cookiePolicy-btn-close>span"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.buy-link.load-more-btn[value='LOAD MORE']"))).click()

  • Using XPATH:

    driver.get("https://www.nvidia.com/en-us/shop/geforce/?page=1&limit=9&locale=en-us")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[@id='cookiePolicy-btn-close']/span"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@class='buy-link load-more-btn' and @value='LOAD MORE']"))).click()

  • Note: You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

  • Browser Snapshot:

LOADMORE

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • Selenium thinks the element was clickable, that's why it attempted the click. The problem is that there was a DIV in the way. See the error message OP posted. – JeffC Jan 12 '21 at 20:57
  • that doesnt work, i dont think its the wait time, I think the element itself is inclickable – Kryptic Coconut Jan 12 '21 at 20:57
  • @JeffC You are wrong, [Selenium](https://stackoverflow.com/questions/54459701/what-is-selenium-and-what-is-webdriver/54482491#54482491) can't think, it's neither a human nor AI enabled robot. The element is a _Angular_ element. So the bottom line is first you have to understand how angular based website specifically angular element behaves and then use Selenium accordingly. – undetected Selenium Jan 12 '21 at 21:08
  • @KrypticCoconut Checkout the updated answer and let me know the status. – undetected Selenium Jan 12 '21 at 21:22
  • @DebanjanB still the same error :( `selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element is not clickable at point (464, 871). Other element would receive the click:
    ...
    `     – Kryptic Coconut Jan 12 '21 at 21:36
  • @KrypticCoconut Did you test copy pasting the same code I've posted? – undetected Selenium Jan 12 '21 at 21:38
  • @DebanjanB yes, exactly the same – Kryptic Coconut Jan 12 '21 at 21:39
  • @KrypticCoconut I still can't believe that. `LOAD MORE` is quite above the intercepting element. Does xpath and Css both gives the same error? – undetected Selenium Jan 12 '21 at 21:42
  • 1
    "Selenium thinks..." is an example of an anthropomorphism. There's nothing magical about Angular elements... if the element wasn't ready to be clicked, it would have thrown a different error. That was my point here and on other of your answers where you've answered in the same way. A wait doesn't fix that... as you've seen and modified your answer now several times. – JeffC Jan 12 '21 at 21:48
  • @KrypticCoconut Did you keep the browser in **maximized** mode while trying to click? – undetected Selenium Jan 12 '21 at 21:55
  • oh, if i maximize it works, sorry if I'm being annoying but why does it do that only when its maximized? – Kryptic Coconut Jan 12 '21 at 22:00
  • @KrypticCoconut Never mind, we are here to help the _annoying_ but innovative minds :) You need to _maximize_ the browser so the desired elements are hosted within the [viewport](https://stackoverflow.com/questions/59126078/how-does-selenium-click-on-elements-that-are-50-on-screen-and-50-not-on-screen/59130869#59130869) – undetected Selenium Jan 12 '21 at 22:02
0

The problem is that there are some 20 elements on that page that match your first locator, .find_element_by_class_name("buy-link"). In this case, if you make your locator more specific, you can isolate it down to just the "LOAD MORE" INPUT button.

Try this instead

driver.find_element_by_css_selector("input.buy-link").click()

It's generally a good practice to add a wait unless you are sure you won't ever need it.

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.XPATH, "//span[text()='CLOSE']")).click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.buy-link")).click()

See the docs for more info and details.

JeffC
  • 22,180
  • 5
  • 32
  • 55
  • i changed the class to`load-more-btn`, that didnt work so I tried yours and same error :\ `selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element is not clickable at point (464, 871). Other element would receive the click:
    ...
    `     – Kryptic Coconut Jan 12 '21 at 21:15
  • So the issue now is that when Selenium scrolls down to click the LOAD MORE button, the button ends up underneath the cookie banner that states, "This site uses cookies to store information on your computer. See our cookie policy for further details on how to block cookies." Once you click CLOSE on the banner, the button should be revealed and clickable. I've updated the code and it successfully runs on my machine. – JeffC Jan 12 '21 at 21:43
  • @JeffC Wrong inception again. Read the error once again, the element `
    ...
    ` is different from the cookie banner element. Why don't you check the website once?     – undetected Selenium Jan 12 '21 at 21:46
  • 1
    @DebanjanB `I've updated the code and it successfully runs on my machine.` You should slow down and actually read the question... it would have saved you having to update your answer several times. You should also slow down and read my comments on your answer because they pointed this out but you out of hand dismissed them as you always do. You should also slow down and actually read *my* answer and the comments below my answer because if you would have, you'd have already seen that I tried this code and it works on my machine. I can't account for someone else's machine settings, etc. – JeffC Jan 12 '21 at 21:51
  • 1
    @DebanjanB Also, I think you mean "assumption", not "inception". [inception definition](https://www.merriam-webster.com/dictionary/inception) – JeffC Jan 12 '21 at 21:52
  • huh, im getting a ctor error ``` driver = webdriver.Chrome('./chromedriver') driver.get("https://www.nvidia.com/en-us/shop/geforce/?page=1&limit=9&locale=en-us") wait = WebDriverWait(driver, 10) wait.until(EC.element_to_be_clickable(By.XPATH, "//span[text()='CLOSE']")).click() wait.until(EC.element_to_be_clickable(By.CSS_SELECTOR, "input.buy-link")).click() ``` `__init__() takes 2 positional arguments but 3 were given` – Kryptic Coconut Jan 12 '21 at 21:52
  • @KrypticCoconut What's the error message? `ctor` is a constructor. – JeffC Jan 12 '21 at 21:54
  • the error was `__init__() takes 2 positional arguments but 3 were given` mentioned it in comment – Kryptic Coconut Jan 12 '21 at 21:57
  • Is it on the `wait` line or which line was it on? – JeffC Jan 12 '21 at 22:00