0

I'm trying to scrape data from the public site. On the search page enter the search value and press the view button (normal submit)its shows the results.

normal browser view button clicks and shows the report as expected. but when I try to click the same button via the Webdriver code, the button clicks but the result was not shown.

Can anyone face this issue?

I tried the button click using XPath(//*[@id="b-form-submit"]/div/button) with the below method

Click()
Submit()
Click using Javascript
Click using action class (keyboard event)

Button HTML code

<div class="b-button" id="b-form-submit">
   <div class="b-button-container">
      <span class="left">
      <i></i>
      </span>
      <button alt="View" type="submit">View</button>
      <span class="right">
      <i></i>
      </span>
   </div>
</div>

When i try to click the view button manually from the webdriver (debug mode) its not wokring, but same its working normal web browser (all the browsers)

Prabu
  • 3,550
  • 9
  • 44
  • 85
  • Is your webpage publicly available? It might be possible website is blocking web scraping. – KunduK Jan 13 '21 at 12:18
  • @KunduK - Yes it's a public webpage. How it's possible to block the web scraping? I can able to navigate the webpage thru the URL and able to enter the search value also – Prabu Jan 13 '21 at 12:23
  • Could you [Edit] your post and update the url so that other contributor can help you. – KunduK Jan 13 '21 at 12:25
  • @KunduK - Sorry due to security reasons I couldn't share the URL. – Prabu Jan 13 '21 at 12:45
  • Try with explicit wait and check whether it works. `new WebDriverWait(driver, 10).until(ExpectedConditions.elementToBeClickable(By.xpath("//button[text()='View']"))).click();` – KunduK Jan 13 '21 at 12:49
  • @KunduK - Yes I used the wait statement but no luck. – Prabu Jan 13 '21 at 13:03

1 Answers1

-1

The <button> isn't an immediate descendant of the <div> you can use either of the following Locator Strategies::

  • css_selectors:

    div#b-form-submit>div button[alt='View']

  • xpath:

    //div[@id="b-form-submit"]/div//button[@alt='View' and text()='View']

So to click() on the element with text as View you can use either of the following Locator Strategies:

  • cssSelector:

    driver.findElement(By.cssSelector("div#b-form-submit>div button[alt='View']")).click();

  • xpath:

    driver.findElement(By.xpath("//div[@id="b-form-submit"]/div//button[@alt='View' and text()='View']")).click();

Ideally to click() on the element you need to induce WebDriverWait for the elementToBeClickable() and you can use either of the following Locator Strategies:

  • cssSelector:

    new WebDriverWait(driver, 20).until(ExpectedConditions.elementToBeClickable(By.cssSelector("div#b-form-submit>div button[alt='View']"))).click();

  • xpath:

    new WebDriverWait(driver, 20).until(ExpectedConditions.elementToBeClickable(By.xpath("//div[@id="b-form-submit"]/div//button[@alt='View' and text()='View']"))).click();
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • I'm not getting any exception with my XPath, and your XPath also correct. and I open the driver EXE load the webpage manually in that driver and click the view button it does not show the results, at the same time I can enter a search value. – Prabu Jan 13 '21 at 14:02
  • @Prabu Checkout the updated answer and let me know the status. – undetected Selenium Jan 13 '21 at 15:43
  • There are no issues in the XPath. The Button clicked in the driver, but not produce the result, I think that site blocked the search result via Driver execution. – Prabu Jan 18 '21 at 07:07
  • @Prabu We aren't aware about the site and not sure if the bot is detected. Can you raise a new question as per your new requirement? Stackoverflow contributors will be happy to help you out. – undetected Selenium Jan 18 '21 at 07:11
  • Thanks Please refer to my new question https://stackoverflow.com/questions/65774494/data-scraping-using-selenium-unable-to-find-the-search-result – Prabu Jan 18 '21 at 12:26
  • did you go through my question, do you have any solution? – Prabu Jan 20 '21 at 11:50