0

I am trying to build some kind selenium scraper for reddit. However, I do find myself having some trouble to get the timestamp shown in the pic.

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.wait import WebDriverWait
from time import sleep # this should go at the top of the file
from bs4 import BeautifulSoup as bs

sleep(5)
driver = webdriver.Chrome(ChromeDriverManager().install())

driver.get('https://www.reddit.com/r/StockMarket/')

title = driver.find_element_by_css_selector('h3._eYtD2XCVieq6emjKBH3m').text
timestamp = driver.find_element_by_css_selector('a._3jOxDPIQ0KaOWpzvSQo-1s').text
print(title)
print(timestamp)

[Output]: We’ve been doing it all wrong
          5 hours ago

Desired result

DisappointedByUnaccountableMod
  • 6,656
  • 4
  • 18
  • 22
Shing Yan Yuen
  • 71
  • 1
  • 6

1 Answers1

0

To get the timestamp (time when content was posted) use css selector:

timestamp = driver.find_element_by_css_selector('a[data-click-id=timestamp]._3jOxDPIQ0KaOWpzvSQo-1s')

See here how to work with dataframes. It's a good example Feed dataframe with webscraping

The locator posts list would be:

post = driver.find_elements_by_css_selector('._1poyrkZ7g36PawDueRza-J._11R7M_VOgKO1RJyRSRErT3 ')

Getting a tooltip locator is a little tricky. You won't see it unless you hover a mouse on timestamp. This works just perfectly: ._2J_zB4R1FH2EjGMkQjedwc.u6HtAZu8_LKL721-EnKuR[style] enter image description here

Check this answer on how to inspect elements on hover Inspect hovered element in Chrome?

vitaliis
  • 4,082
  • 5
  • 18
  • 40
  • But how can I inspect the element if I need to hover the mouse on it? Can you explain it in detail? Thank you. – Shing Yan Yuen May 02 '21 at 06:46
  • I have tried but this error pops up. date = driver.find_element_by_class_name('_2J_zB4R1FH2EjGMkQjedwc u6HtAZu8_LKL721-EnKuR').text NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".div._2J_zB4R1FH2EjGMkQjedwc u6HtAZu8_LKL721-EnKuR"} (Session info: chrome=90.0.4430.93) – Shing Yan Yuen May 02 '21 at 07:10
  • right now I can find the proper class name and xpath of the tooltip text, but I cannot access the text with neither the find_element_by_class_name or find_element_by_xpath method. The following error pops up: NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[7]/div/text()"} (Session info: chrome=90.0.4430.93) – Shing Yan Yuen May 02 '21 at 09:41
  • You won't access it without hovering mouse on it. – vitaliis May 02 '21 at 13:30