How to crawl the title of the page?

Question

I don't know how to crawl the title of the page,below is my code(it's simple),but I have no idea where is wrong, if you have any idea please let me know,thank you.

from selenium import webdriver
url="https://sukebei.nyaa.si/?s=seeders&o=desc&p=1"
driver_path = "C:\\webdriver\\chromedriver.exe"
option = webdriver.ChromeOptions()
driver = webdriver.Chrome(driver_path, options=option)
driver.implicitly_wait(10)
driver.get(url)
print(driver.find_element_by_xpath("/html/head/title").text)

driver.title returns title – Arundeep Chohan Feb 13 '21 at 16:24 — Arundeep Chohan, Feb 13 '21 at 16:24

score 2 · Accepted Answer · answered Feb 13 '21 at 17:19

To crawl the title of the page you have to induce WebDriverWait for the visibility_of_element_located() for the <table> with torrent-list and you can use either of the following Locator Strategies:

Using CSS_SELECTOR:

driver.get('https://sukebei.nyaa.si/?s=seeders&o=desc&p=1')
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "table.torrent-list")))
print(driver.title)

Using XPATH:

driver.get('https://sukebei.nyaa.si/?s=seeders&o=desc&p=1')
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//table[contains(@class, 'torrent-list')]")))
print(driver.title)

Console Output:
```
Browse :: Sukebei
```

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

score 1 · Answer 2 · answered Feb 13 '21 at 16:53

from selenium import webdriver
url="https://sukebei.nyaa.si/?s=seeders&o=desc&p=1"
driver_path = "C:\\webdriver\\chromedriver.exe"
option = webdriver.ChromeOptions()
driver = webdriver.Chrome(driver_path, options=option)
driver.implicitly_wait(10)
driver.get(url)
print(driver.title)

How to crawl the title of the page?

2 Answers2

Linked