1

I have a little problem, but because im absolutely beginner, i cant find the answers... Could you help me?

So, I run a little realtor business, and I would like to scrape the local ads on FB Marketplace.

But for some reason, I can't progress, when I need to use xpath, to find elements get this:

 raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="mount_0_0_/O"]/div/div[1]/div/div[4]/div/div/div[1]/div/div[3]/div[2]/div/div[2]/div/div[2]/div/div[2]/div[1]/div[1]/span/text()"}
  (Session info: chrome=97.0.4692.71)

So i have the following questions:

  • First of all, what the heck am i doing wrong? How can i solve this problem?
  • What do you suggest, how can I improve/debug the code i have done until now?
  • What is your suggestion, how should I improve my skills more efficiently in python?

Thank you for your help :)

# imports
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait
# from selenium.webdriver.common.keys import Keys
import time
# import csv

# Exceptions to handle unexpected faults!!!
# from selenium.common.exceptions import WebDriverException
# from selenium.common.exceptions import NoSuchElementException
# from selenium.common.exceptions import NoSuchFrameException
# from selenium.common.exceptions import NoSuchWindowException
# from selenium.common.exceptions import ElementNotInteractableException
# from selenium.common.exceptions import StaleElementReferenceException
# from selenium.common.exceptions import MoveTargetOutOfBoundsException
# from selenium.common.exceptions import UnexpectedAlertPresentException
# from selenium.common.exceptions import ElementClickInterceptedException
# from selenium.common.exceptions import JavascriptException


chrome_options = webdriver.ChromeOptions()
prefs = {"profile.default_content_setting_values.notifications" : 2}
chrome_options.add_experimental_option("prefs",prefs)
driver = webdriver.Chrome(chrome_options=chrome_options)

driver = webdriver.Chrome('F:/Real-Trader/chromedriver', chrome_options=chrome_options)

driver.get("http://www.facebook.com")

alert=WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.XPATH, '//button[contains(text(), "Az összes cookie engedélyezése")]'))).click()
username = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[name='email']")))
password = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[name='pass']")))

username.clear()
username.send_keys("@@@@@@@@@@@@@")
password.clear()
password.send_keys("*************")

button = WebDriverWait(driver, 2).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[type='submit']"))).click()
 
time.sleep(5) 

driver.get("https://www.facebook.com/marketplace/109375022413858/propertyforsale/")
time.sleep(5)
    
for j in range(0,0):
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    time.sleep(5)

anchors = driver.find_elements_by_tag_name('a')
anchors = [a.get_attribute('href') for a in anchors]
   
anchors = [a for a in anchors if str(a).startswith('https://www.facebook.com/marketplace/item/')]

links = []

for a in range(len(anchors)):
    #print('------------------------')
    the_link = str(anchors[a])
    character_cut = int(anchors[a].find('/?'))
    cut_link = the_link[:character_cut]
    #print(cut_link)
    links.append(cut_link)
    
ad = []

for elem in links:
    print('------------------------')
    print(elem)
    driver.get(elem)
    time.sleep(5)
    
# This is, what i should use succesfully....!
    title = driver.find_element_by_xpath('//*[@id="mount_0_0_/O"]/div/div[1]/div/div[4]/div/div/div[1]/div/div[3]/div[2]/div/div[2]/div/div[2]/div/div[2]/div[1]/div[1]/span/text()').text
    location = driver.find_element_by_xpath('//*[@id="mount_0_0_/O"]/div/div[1]/div/div[4]/div/div/div[1]/div/div[3]/div[2]/div/div[2]/div/div[2]/div/div[2]/div[1]/div[7]/div[3]/div/div[1]/span/text()').text
    price = driver.find_element_by_xpath('//*[@id="mount_0_0_/O"]/div/div[1]/div/div[4]/div/div/div[1]/div/div[3]/div[2]/div/div[2]/div/div[2]/div/div[2]/div[1]/div[1]/div[1]/div/span/text()').text
    button = WebDriverWait(driver, 2).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[type='Tovább']"))).click()
    description = driver.find_element_by_xpath('//*[@id="mount_0_0_/O"]/div/div[1]/div/div[4]/div/div/div[1]/div/div[3]/div[2]/div/div[2]/div/div[2]/div/div[2]/div[1]/div[8]/div[2]/div/div/div/span/text()').text
    seller_name = driver.find_element_by_xpath('//*[@id="mount_0_0_/O"]/div/div[1]/div/div[4]/div/div/div[1]/div/div[3]/div[2]/div/div[2]/div/div[2]/div/div[2]/div[1]/div[9]/div/div/div[2]/div/div/div/div/div[2]/div/div/div/div[1]/span/span/div/div/div/span').text
    button = WebDriverWait(driver, 2).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[type='Eladó adatai']"))).click()
    # seller_data = driver.find_element_by_xpath().text
    
# This is, what is actually working... :(
    # description = driver.find_element_by_tag_name('div')
    # print(description.text)
    # ad.append(description)
  • include the markup of the element you are targeting here. You'll want to find what is unique about it and target that. (Usually ID would be enough as that should be unique...) Using long path-based selectors is always a bad idea. Only use path-based notation for parts of the selector and only when absolutely necessary. It's also a good idea to always use webdriverwaits. (Though after a .get(url) you should be fine without... you won't even need that sleep there as Selenium will wait for full page load. Only exception is lazy-loading sites) – pcalkins Jan 14 '22 at 22:58

0 Answers0