0

I am trying to use Selenium to scrape a e-commerce website www.flipkart.com Here are the steps I am trying for:

1. Open Flipkart
2. Close the pop-up login prompt
3. Go to the Search Menu
4. Search for a keyword "Laptop"8/*/-
5. Click on the Search button
6. Open every record as a new tab and scrape information from it.
7. After scraping close the newly opened tab and go to the parent tab.
8. Repeat Steps for all the records on the page.
9. After getting all the records from one page go to the next page and repeat steps.

The problem I have with my code is that I am unable to close the newly opened tab from step 6.

driver=webdriver.Chrome()
driver.get("http://www.flipkart.com")

#Close pop us
elem=driver.find_element_by_xpath("/html/body/div[2]/div/div/button")
elem.click()

#click on search field
#1
elem=driver.find_element_by_xpath("xpath_tag_too_long_to_add")
elem.send_keys("Laptop")

#search
#2
elem=driver.find_element_by_xpath("xpath_tag_too_long_to_add")
elem.click()
time.sleep(10)

#get the maximum page number
y=int(driver.find_element_by_css_selector('div._2zg3yZ').text[10:12])

#scrape pages
name=[]
price=[]
color=[]
for i in range(1,y+1):
    driver.get(link+str(i))
    x=driver.find_elements_by_css_selector('div._3wU53n')
    for i in x:
       i.click()
       actions = ActionChains(driver)      
       driver.switch_to.window(driver.window_handles[1])
       time.sleep(2)
       #3
       names=driver.find_element_by_xpath('xpath_tag_too_long_to_add').text
       name.append(name)
       #4
       prices=driver.find_element_by_xpath('xpath_tag_too_long_to_add').text
       price.append(prices)
       #5
       colors=driver.find_element_by_xpath('xpath_tag_too_long_to_add').text
       color.append(colors)
       driver.switch_to.window(driver.window_handles[0])

I have tried things like action keys to close a tab, but it has not worked. I problem I get because of not closing the newly opened tab is that when I use switch_to.window it keeps going back and forth with the parent page and the first opened page giving me the same value in all the columns.

      #this is what I have tried
      elem=driver.find_element_by_tag_name("body")
      elem.send_keys(Keys.CONTROL+"w")


     Here are the Xpath & Links:
     #1-/html/body/div[1]/div/div[1]/div[1]/div[2]/div[2]/form/div/div/input
     #2-/html/body/div[1]/div/div[1]/div[1]/div[2]/div[2]/form/div/button
     #3-/html/body/div[1]/div/div[3]/div[2]/div[1]/div[2]/div[2]/div/div[1]/h1/span
     #4-/html/body/div[1]/div/div[3]/div[2]/div[1]/div[2]/div[2]/div/div[3]/div[1]/div/div[1]
     #5-/html/body/div[1]/div/div[3]/div[2]/div[1]/div[2]/div[7]/div[3]
        /div/div[2]/div[1]/div[1]/table/tbody/tr[5]/td[2]/ul/li

link-https://www.flipkart.com/search?q=laptop&otracker=search&otracker1=search&marketplace=FLIPKART&as-show=on&as=off&page=

Link to Website: https://www.flipkart.com/

Thanks for reading.

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
Nitin Kumar
  • 69
  • 1
  • 6
  • Please take a look on this post: https://stackoverflow.com/a/39037983/3079726. It should answer your question. – mchfrnc Jun 07 '20 at 10:43

1 Answers1

-1

Once you open the Flipkart using get(), store the parent window handle, i.e. current_window_handle

driver.get("https://www.flipkart.com")
parent_window  = driver.current_window_handle

Next within the for() loop once you append the colors, close() the adjascent new tab and then switch to the parent window as follows:

for i in x:
    #previous lines of code
    #5
    color.append(colors)
    driver.close()
    driver.switch_to.window(parent_window)

You can find a relavent discussion in Selenium Switch Tabs

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352