0

In the try block where I have written "#print text details of table 1:" is where I am stuck.
I am looking to extract the text from the 1st table in the popup upon clicking "details". XPATH: /html/body/table[1].

driver.get("https://www.drayage.com/directory/results.cfm?city=SAV&port=y&OceanCntrs=y&drvrs=y&showClicks=y")
trs = wait.until(EC.visibility_of_all_elements_located((By.XPATH,"//html/body/table/tbody/tr/td/table[1]//tr[position()>2]")))
window_before = driver.window_handles[0]
for tr in trs:
    try:
        detail = tr.find_element(By.XPATH,".//a[contains(.,'detail')]")
        detail.click()
        wait = WebDriverWait(driver,10)
        
        #Handle the tab switch
        window_after = driver.window_handles[1]
        driver.switch_to.window(window_after)
        sleep(2)
        
        #Print text details of table 1:

        #Continue iterating through "detail" popups
        driver.close()
        driver.switch_to.window(window_before)
    except:
        print("No detail")

I attempted to iterate through the table body and append the text to a list and print the list but my lists are coming back empty.

        #Print text details of table 1:
        my_list = []
        new_text=driver.find_elements_by_xpath(("//table[@id='dataLstSubCat']/tbody/tr/td"))
        for text in new_text:
            my_list.append(text.text)
        print(my_list)

Here are the imports you will need for my code:

#import packages
from selenium import webdriver
from time import sleep
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC
Tyler Simpson
  • 13
  • 3
  • Please note this is not a code-writing service - give a [mre] (is all of the scraping code relevant at all?) showing what you've tried and where the problem is. – jonrsharpe Feb 20 '22 at 17:05
  • @jonrsharpe I went ahead and made it more concise, sorry about that. Hopefully this fits better. My problem is grabbing the info (all the rows) in the table[1] of the popup because it has no IDs and I am still quite new. – Tyler Simpson Feb 20 '22 at 17:13
  • 1
    Please re-read the linked article. Again, is how you scrape that HTML relevant, or could you start later in the process? Also `#Extract info?` is not an approrpiate amount of effort to put into an actual implementation. – jonrsharpe Feb 20 '22 at 17:17
  • @jonrsharpe thanks again for the input. I tried to improve my code and question again in line with the article you linked. I am not sure what you mean by "is how you scrape that HTML relevant, or could you start later in the process?". I suppose my thought process was that it would help better frame the issue. I can certainly remove it, though. – Tyler Simpson Feb 20 '22 at 17:33
  • All you've done now is move some imports and claim you've tried some things - OK, so show one, help us understand exactly where you're stuck. And the scraping method isn't relevant if we can start from the string of HTML you'd receive, which also shows us the structure. – jonrsharpe Feb 20 '22 at 17:35
  • 1
    @TylerSimpson To start with `window_after = driver.window_handles[1]` isn't the ideal way to switch [tabs](https://stackoverflow.com/a/51893230/7429447). Possibly the [_focus_](https://stackoverflow.com/a/59521886/7429447) haven't changed. – undetected Selenium Feb 20 '22 at 17:56
  • @jonrsharpe my apologies I understand the confusion. I am hoping each edit makes it clearer and I appreciate your input. – Tyler Simpson Feb 20 '22 at 18:04
  • 1
    @undetectedSelenium Thank you! I had not thought about the focus not changing. I will work on this now! – Tyler Simpson Feb 20 '22 at 18:04

1 Answers1

0

Everyone thank you so much for your support on this!

Thanks to help from the comments I was able to answer this question.

I had to change the xpath in the code block "#Print text details of table 1:".

Answer:

        #Print text details of table 1:
        my_list = []
        new_text=driver.find_elements_by_xpath(("/html/body/table[1]"))
        for text in new_text:
            my_list.append(text.text)
        print(my_list)

Original:

        #Print text details of table 1:
        my_list = []
        new_text=driver.find_elements_by_xpath(("//table[@id='dataLstSubCat']/tbody/tr/td"))
        for text in new_text:
            my_list.append(text.text)
        print(my_list)
Tyler Simpson
  • 13
  • 3