1

i am using Beutifull soup to scrape this web page: https://greyhoundbet.racingpost.com//#results-dog/race_id=1765914&dog_id=527442&r_date=2020-03-19&track_id=61&r_time=11:03

Result: i get the javaScript, Css

Desired output: i need the main html

i used this code

import requests
from bs4 import BeautifulSoup 

url = 'https://greyhoundbet.racingpost.com//#results-dog/race_id=1765914&dog_id=527442&r_date=2020-03-19&track_id=61&r_time=11:03'

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36'}

page = requests.get(url,headers=headers)url = 'https://greyhoundbet.racingpost.com//#results-dog/race_id=1765914&dog_id=527442&r_date=2020-03-19&track_id=61&r_time=11:03'

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36'}

page = requests.get(url,headers=headers)

soup = BeautifulSoup(page.content, 'html.parser')
BenPortner
  • 43
  • 7
Mostafa Ahmed
  • 23
  • 5

2 Answers2

0

I think what you looking for is this:

page = requests.get(url)

soup = BeautifulSoup(page.text, 'html.parser')

It will contain the text from the page including html tags

Hendrik Evert
  • 340
  • 2
  • 19
0

I’m afraid you won’t be able to get it directly using BeautifulSoup because the page loads then a javascript loads data.

It’s one of the component’s limitations, you may need to use selenium.

please check the answers on this question

Mostafa Abu EL-Khair
  • 16