I am trying to take data from a tech site but when I try to print productBox variable output is showing this: [ ] Can someone solve it
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('https://www.itopya.com/bilgisayar/masaustu/oyun-bilgisayarlari/')
data = driver.find_element_by_id('allProducts')
productBox = data.find_elements_by_class_name('product col-md-3')
print(productBox)
To print the product names you have to induce WebDriverWait for the visibility_of_all_elements_located() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
driver.get("https://www.itopya.com/bilgisayar/masaustu/oyun-bilgisayarlari/?__cf_chl_jschl_tk__=f128b939cc717f7b22114e1e0f850e5b3cf16d59-1594571020-0-Aaj8nKhAOskxwytnAGUERyGvp77H0pQGK05Lmlkoq0ec5KDpM_1dDn_QLd7n-RueS6-HeAUXEbxqUIGNrGxcavD3O5G58hK-0BiQRnZVV6kr4lOBK7zHG7hMbUmjfjmYL_U_T7MKMi8rzaKgpnW5q9u4x-75lLaO8hf075M_x7788TNXBhsWuFFCxCOmBEqBrxV4J7klhj9tGaU1vYtpqY_ZDKtWuV8EkHYwswACUqLO53ddGwIjI2x1Uu22KxFy9XdvcE47dBPBp1pzcpwyZHnap625G7vxmIoZBMlq1JEgCIWUu-FMqpPlGhTpxigJwCkHaIvqcW9RlkvTbKo9PsruvjCIQFqEjD2JQSwX3LQzVVwmipkD2GNOjsLsQ6ypRKYwWSZ07AOzLuKny9mqvOQ")
print([my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(driver, 5).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.product-inner div.product-title>a")))])
Using XPATH:
driver.get("https://www.itopya.com/bilgisayar/masaustu/oyun-bilgisayarlari/?__cf_chl_jschl_tk__=f128b939cc717f7b22114e1e0f850e5b3cf16d59-1594571020-0-Aaj8nKhAOskxwytnAGUERyGvp77H0pQGK05Lmlkoq0ec5KDpM_1dDn_QLd7n-RueS6-HeAUXEbxqUIGNrGxcavD3O5G58hK-0BiQRnZVV6kr4lOBK7zHG7hMbUmjfjmYL_U_T7MKMi8rzaKgpnW5q9u4x-75lLaO8hf075M_x7788TNXBhsWuFFCxCOmBEqBrxV4J7klhj9tGaU1vYtpqY_ZDKtWuV8EkHYwswACUqLO53ddGwIjI2x1Uu22KxFy9XdvcE47dBPBp1pzcpwyZHnap625G7vxmIoZBMlq1JEgCIWUu-FMqpPlGhTpxigJwCkHaIvqcW9RlkvTbKo9PsruvjCIQFqEjD2JQSwX3LQzVVwmipkD2GNOjsLsQ6ypRKYwWSZ07AOzLuKny9mqvOQ")
print([my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(driver, 5).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='product-inner']//div[contains(@class, 'product-title')]/a")))])
Console Output:
['MSI MAG META 5 3SC-003XTR R5-3600 16GB DDR4 512GB SSD+1TB HDD RTX2060 SUPER GDDR6 8GB DOS Gaming Bilgisayar', 'MSI TRIDENT 3 9SI-614EU I5-9400F 16GB DDR4 512GB SSD GTX1660 SUPER GDDR6 6GB W10 Gaming Bilgisayar', 'MSI AEGIS 3 9SC-236EU I7-9700F 16GB DDR4 512GB SSD+1TB HDD RTX2060 SUPER GDDR6 8GB W10 Gaming Bilgisayar', 'HERMES-A3660SX/AMD Ryzen 5 3600/ASUS TUF 3 GTX 1660 SUPER OC 6GB/16GB DDR4/240GB SSD/1TB Disk/Gaming Bilgisayar', 'Powered by ICUE-M10980KT/MSI RTX 2080 Ti GAMING Z TRIO 11GB/INTEL i9 10900K/32GB DDR4/2TB NVMe SSD/2TB DISK/Geforce RTX RayTracing PC', 'ATLAS-M3656XT/AMD Ryzen 5 3600XT/MSI RX 5600 XT MECH OC 6GB/16GB DDR4/500GB NVMe M.2 SSD/1TB DISK/Gaming Bilgisayar', 'EASTERGAMERS/ASUS ROG STRIX RTX 2060 Advanced Edition 6GB/AMD Ryzen 5 3600/16GB DDR4/500GB NVMe M.2 SSD/Geforce RTX RayTracing PC', 'TANK-A10460/ASUS ROG STRIX GTX 1660 SUPER Advanced Edition 6GB/Intel i5 10400/16GB DDR4/480GB SSD/1TB DISK/Geforce Espor PC', 'VESTA-A9656KT/Intel i5 9600K/ASUS TUF 3 Radeon RX 5600 XT OC EVO 6GB/16GB DDR4/240GB SSD/1TB DISK/Gaming Bilgisayar', 'HERMES-A3556XT/AMD Ryzen 5 3500X/ASUS TUF 3 RX 5600 XT OC EVO 6GB/16GB DDR4/1TB DISK/Gaming Bilgisayar', 'DA-SISTEM2/AMD Ryzen 5 3600/ASUS TUF 3 RX 5600 XT OC EVO 6GB/16GB DDR4/480GB SSD/Gaming Bilgisayar', 'WTCN-NUKE/ASUS TUF GTX 1650 SUPER OC 4GB/INTEL i3 10100/8GB DDR4/480GB SSD/Geforce Espor PC', 'VOK-IMPERIAL/AMD Ryzen 5 3500X/MSI RX 5600 XT MECH OC 6GB/16GB DDR4/480GB SSD/Gaming Bilgisayar', 'MARS-A1060K/ASUS ROG STRIX RTX 2060 Advanced Edition 6GB/INTEL i7 10700K/16GB DDR4/480GB SSD/2TB DISK/Geforce RTX RayTracing PC', 'ALPER RENDE-ANDROMEDA/AMD Ryzen 5 3600/ASUS ROG STRIX RX 5700 OC 8GB/16GB DDR4/480GB SSD/Gaming Bilgisayar', 'ALPER RENDE-CARRY/ASUS ROG STRIX RTX 2060 OC 6GB/AMD Ryzen 5 3600/16GB DDR4/480GB SSD/Geforce RTX RayTracing PC', 'RENDE-SUPREME/ASUS ROG STRIX GTX 1660 SUPER OC 6GB/AMD Ryzen 5 3500X/16GB DDR4/480GB SSD/Geforce Espor PC', 'TEKNOKING-2M/AMD Ryzen 5 3500X/MSI RX 5700 MECH GP OC 8GB/16GB DDR4/480GB SSD/Gaming Bilgisayar', 'MSI INFINITE X PLUS 9SE-297EU I7-9700K 16GB DDR4 512GB SSD 2TB 7200RPM HDD RTX2080 GDDR6 8GB W10 DVD Gaming Bilgisayar', 'MSI INFINITE X PLUS 9SD-450TR I7-9700K 16GB DDR4 256GB SSD 2TB 7200RPM HDD RTX2070 GDDR6 8GB W10 DVD Gaming Bilgisayar']
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC