I am trying to look at a couple of websites https://sslmate.com/labs/crl_watch/ and https://sslmate.com/labs/ocsp_watch/ and I'm trying to see if a specific keyword shows up in the table. That's all I need to know. For example sake I want to take a look to see if GoDaddy shows up. I've used this code
page = requests.get("https://sslmate.com/labs/ocsp_watch/").text
print(page)
print(re.findall("GoDaddy", page))
print(page.find("GoDaddy"))
But it seems like it's printing just the main headers and not what's actually within the table. This is my first time looking into parsing web pages, so it's probably something simple that I'm missing. I came across this link Extracting data from HTML table and another one about gathering a list of how many times the word football appears searching fifa's page but I can't seem to find that url at the moment. When I run the Inspect Element here is the specific elements I'm trying to search in.
I also took an attempt of using BeautifulSoup and here is my code for that. This code was based on another stack over flow question I saw. And I changed it to my needs but didn't work as expected :D
from bs4 import BeautifulSoup as bs
temp = urllib.request.urlopen('https://sslmate.com/labs/crl_watch/')
HTML = temp.read().decode("utf-8")
soup = bs(HTML)
table = soup.find(table = soup.find("table", attrs={"id":"operator problems"}))
headings = [th.get_text() for th in table.find("tr").find_all("th")]
datasets = []
for row in table.find_all("tr")[1:]:
dataset = headings, (td.get_text() for td in row.find_all("td"))
datasets.append(dataset)
print(datasets)
Above I was just trying to print what was in the specific heading and it seems to be almost there but the objects are being printed instead of the values.
