0

I had asked similar questions some time back and I am still facing errors trying to parse a webpage. The scenario is that the system navigates to https://shop.sprouts.com/shop/flyer and wants to extract each of the specials related to each category. Currently when a category is clicked, I see a blank screen on the right hand side. I have tried both the options below but I get the same results.

elementLeftHandSideMenu.Click();
iJavaScriptExecutor.ExecuteScript("arguments[0].click();", elementLeftHandSideMenu);

Is this a timing issue? What am I doing wrong?

using Microsoft.VisualStudio.TestTools.UnitTesting;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System;
using System.IO;
using System.Linq;
using System.Reflection;
using SeleniumExtras.WaitHelpers;
using System.Threading;
using System.Collections.Generic;
using Newtonsoft.Json;

[TestClass]
public class UnitTest1
{
ChromeDriver driver;
WebDriverWait webDriverWait;
[TestInitialize]
public void Startup()
{
  var chromeOptions = new ChromeOptions();
  chromeOptions.AddArguments("--proxy-server='direct://'");
  chromeOptions.AddArguments("--proxy-bypass-list=*");
  chromeOptions.AddArguments("--start-maximized");

  var path = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
  ChromeDriverService chromeDriverService = ChromeDriverService.CreateDefaultService(path);
  driver = new ChromeDriver(chromeDriverService, chromeOptions);
  webDriverWait = new WebDriverWait(driver, TimeSpan.FromSeconds(30));
}
[TestCleanup]
public void CleanUp()
{
  driver.Quit();
}
[TestMethod]
public void GetSproutsWeeklyAdDetails2()
{
  try
  {
    driver.Navigate().GoToUrl("https://shop.sprouts.com/shop/flyer");
  }
  catch (TimeoutException timeoutException)
  {
    driver.Navigate().Refresh();
  }
  var iJavaScriptExecutor = (IJavaScriptExecutor)driver;
  webDriverWait.Until(driver1 => iJavaScriptExecutor.ExecuteScript("return document.readyState").Equals("complete"));

  var elementsLeftHandSideMenu = webDriverWait.Until(ExpectedConditions.VisibilityOfAllElementsLocatedBy(
    By.XPath("//ul[@class='menu sidenav']/li[@class='child']/a")));
  System.Diagnostics.Debug.WriteLine(elementsLeftHandSideMenu.Count);
  var items = new List<Item>();
  foreach (var elementLeftHandSideMenu in elementsLeftHandSideMenu)
  {
    Thread.Sleep(2000);
    try
    {
      elementLeftHandSideMenu.Click();
      //iJavaScriptExecutor.ExecuteScript("arguments[0].click();", elementLeftHandSideMenu);
    }
    catch (Exception exception)
    {
      System.Diagnostics.Debug.WriteLine(exception.ToString());
    }
    //parsing code here does not work as the RHS shows a blank page
    System.Diagnostics.Debug.WriteLine("Exit");
  }
}
}

enter image description here

Ajit Goel
  • 4,180
  • 7
  • 59
  • 107

1 Answers1

1

It doesn't seem a problem with your element click. Your website seems to have a problem. Even if I click manually, it doesn't load the first time. You can try moving driver.Navigate().Refresh(); from catch to try itself i.e. refresh page once its opened or may be after first click. I am sure it will work.

try
  {
    driver.Navigate().GoToUrl("https://shop.sprouts.com/shop/flyer");
    driver.Navigate().Refresh();
  }
  catch (TimeoutException timeoutException)
  {
    driver.Navigate().Refresh();
  }

If the above code doesn't work, try -

try
    {
      elementLeftHandSideMenu.Click();
      driver.Navigate().Refresh();
      //iJavaScriptExecutor.ExecuteScript("arguments[0].click();", elementLeftHandSideMenu);
    }
eduPeeth
  • 1,840
  • 13
  • 21
  • Adding driver.Navigate().Refresh() after elementLeftHandSideMenu.Click() fixed the original issue but i get a "OpenQA.Selenium.StaleElementReferenceException: stale element reference: element is not attached to the page document" error when it goes to the second element in the foreach loop. Any inputs on what I need to do here to fix this issue? – Ajit Goel Jul 19 '18 at 01:45
  • Probably, you need to find the same element in a for loop 2-3 times, break the loop once you get it. Put the whole thing in try catch and catch the Stale element exception. You can refer [here](https://stackoverflow.com/questions/18225997/stale-element-reference-element-is-not-attached-to-the-page-document) . – eduPeeth Jul 19 '18 at 04:39