Scrape html page that has text embedded in stylesheet and woff file

Question

I want to scrape a webpage but some data is embedded in the stylesheet and woff files.

Here are the links https://777codes.com/newtestament/mat1.html I want the Greek text here which does not show at all in Chromes inspector

And from here https://777codes.com/newtestament/gen1.html I want to get the Hebrew text but if you look in Chromes inspector you will see some "???" which comes out in the scrape

Basically Chromes element inspector shows blank or question marks but it shows correctly in the browser so I know the data is there.

Data missing is in Greek and Hebrew language.

I tried some basic scrapes with Beautiful Soup and very simple Selenium. They give the data in the element inspector which is incorrect. I want to get what I see in the browser.

I understand that sometimes Javascript renders content but this is a bit different I think.

Answer 1

Actually, you don't need the transliterate library. I was able to extract the hebrew chars from the site using beautiful soup.

import requests
from bs4 import BeautifulSoup

page = requests.get("https://777codes.com/newtestament/gen1.html")
soup = BeautifulSoup(page.content, "html.parser")

first_hebrew_word = soup.find("div", class_="stl_01 stl_21")

# outputs 1:1 יתꢀרא (including hebrew chars)
print(first_hebrew_word.text)

# if you want to clean the output

# copy the object to prevent future errors
word = first_hebrew_word.__copy__()
for garbage in word.find_all("span", class_="stl_22"):
    # remove garbage
    garbage.decompose()

# outputs יתꢀראꢁ (including hebrew chars)
print(word.text.strip())

with open("output.txt", "w") as file:
    file.write(word.text.strip() + "\n")

Output text in gedit (ubuntu linux)

Zoomed output in firefox (ubuntu linux)