Getting a number from pubchem site with Selenium

Question

I'm doing a search on the pubchem site with the code below. I need to get the "Compound CID:" number from the screen from the search result but I couldn't get it. I need help on this.

driver = webdriver.Chrome()
url = "https://pubchem.ncbi.nlm.nih.gov/"
driver.get(url)
driver.maximize_window()
searchInput = driver.find_element_by_xpath("/html/body/div[1]/div/div/main/div[1]/div/div[2]/div/div[2]/form/div/div[1]/input")
searchInput.click()
searchInput.send_keys("75-05-8")
searchInput.send_keys(Keys.ENTER)
time.sleep(2)
driver.close()

undetected Selenium · Accepted Answer · 2022-01-01T07:39:53.793

To print the text 6342 you can use either of the following Locator Strategies:

Using css_selector and get_attribute("innerHTML"):

print(driver.find_element(By.CSS_SELECTOR, "a[data-label^='Featured Compound Result Secondary Link; Position:1; Page:1'] > span.breakword > span").get_attribute("innerHTML"))

Using xpath and text attribute:

print(driver.find_element(By.XPATH, "//a[starts-with(@data-label, 'Featured Compound Result Secondary Link; Position:1; Page:1')]/span[@class='breakword']/span").text)

Ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:

Using CSS_SELECTOR and text attribute:

driver.get("https://pubchem.ncbi.nlm.nih.gov/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[type='text'][id^='search']"))).send_keys("75-05-8" + Keys.RETURN)
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "a[data-label^='Featured Compound Result Secondary Link; Position:1; Page:1'] > span.breakword > span"))).text)

Using XPATH and get_attribute("innerHTML"):

driver.get("https://pubchem.ncbi.nlm.nih.gov/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@type='text'][starts-with(@id, 'search')]"))).send_keys("75-05-8" + Keys.RETURN)
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//a[starts-with(@data-label, 'Featured Compound Result Secondary Link; Position:1; Page:1')]/span[@class='breakword']/span"))).text)

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Console Output:
```
6342
```

You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

References

Link to useful documentation:

get_attribute() method Gets the given attribute or property of the element.
text attribute returns The text of the element.
Difference between text and innerHTML using Selenium

Thank you so much. This is very important for me. I have an intermediate level of knowledge. I will look at the documents you provided. Thank you for your detailed answer. — Burak Muhammed ÖNER, Jan 01 '22 at 07:41
I mean you used time.sleep() and a depreciated driver.find_by_xpath() also you don't seem to have Service as well. — Arundeep Chohan, Jan 01 '22 at 08:33
@ArundeepChohan Where did ya find _`time.sleep()`_ and _`driver.find_by_xpath()`_ in my answer :/ — undetected Selenium, Jan 01 '22 at 08:35

Getting a number from pubchem site with Selenium

1 Answers1

References