Scraping a React-table using Selenium

Question

I have written a code for scaping an HTML React table by using python selenium. But it cannot catch values in the table(only DOM elements). Here is the website https://nonfungible.com/market/history/decentraland?filter=saleType%3D&length=10&sort=blockTimestamp%3Ddesc&start=0 Here is my code:

from selenium import webdriver
dr = webdriver.PhantomJS(r'PATH_TO_PHANTOM/phantomjs-2.1.1-macosx/bin/phantomjs')
dr.get("https://nonfungible.com/market/history/decentraland?filter=saleType%3D&length=10&sort=blockTimestamp%3Ddesc&start=0")
table = dr.find_elements_by_class_name("rt-tr-group")
for item in table:
    headers = ",".join([th.text for th in item.find_elements_by_class_name("rt-td")])
    print(headers)

It just prints nothing.

Thank you for helping!

Answer 1

If you intend to use ChromeDriver and google-chrome you can use the following Locator Strategy:

• Code Block:

  driver.get("https://nonfungible.com/market/history/decentraland?filter=saleType%3D&length=10&sort=blockTimestamp%3Ddesc&start=0")
  print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.rt-tr-group div.rt-td")))])
  

• Console Output:

  ['LAND', '115792089237316195423570985008687907834894736851909887012435355206597646221309', '', '', '', '$798.462', '20,000 (MANA)', '1 hr ago 7/14/2020, 12:23:58 PM', 'DCLENS', '43469741616618090506261857019714019688977185639939981493731790027408795972144', '', '', '', '$13.141', '333 (MANA)', '3 hrs ago 7/14/2020, 10:48:46 AM', 'DCLENS', '89838368294314845493020349053893353532486044687731983038462074466093608387295', '', '', '', '$11.627', '295 (MANA)', '3 hrs ago 7/14/2020, 10:44:24 AM', 'DCLENS', '25527909519625043332149265050323189196974713653740181118935970777552699458072', '', '', '', '$4.035', '100 (MANA)', '5 hrs ago 7/14/2020, 8:23:03 AM', 'DCLXMAS2019', '349', '', '', '', '$50.91', '1,251 (MANA)', '7 hrs ago 7/14/2020, 6:11:31 AM', 'LAND', '1361129467683753853853498429727072845789', '', '', '', '$897.357', '22,000 (MANA)', '9 hrs ago 7/14/2020, 4:48:01 AM', 'DCLLNCH', '3629', '', '', '', '$8.164', '200 (MANA)', '10 hrs ago 7/14/2020, 3:04:35 AM', 'LAND', '115792089237316195423570985008687907837276713420356456256678977458620023701615', '', '', '', '$609.161', '14,960 (MANA)', '11 hrs ago 7/14/2020, 2:38:29 AM', 'DCLENS', '93012029374970918876221125714905812452590679293106342049263811906231821801921', '', '', '', '$4.086', '100 (MANA)', '12 hrs ago 7/14/2020, 1:39:25 AM', 'DCLMNSHT2020', '2260', '', '', '', '$19.138', '0.08 (ETH)', '13 hrs ago 7/14/2020, 1:00:43 AM']