0

I'm trying to scrape several websites with using requests package / API calls. Now I'm trying to scrape this website : https://sport.circus.be/en/sport/sports-bets/844/227875758 but this is using AJAX Calls. This seems very unclear to me how I can scrape the data from this (without using Selenium or something) so I can convert it to some JSON / Dataframe in my project.

Can someone help me out please? Thanks a lot!

Stef Renneboog
  • 1
  • Does this answer your question? [Web-scraping JavaScript page with Python](https://stackoverflow.com/questions/8049520/web-scraping-javascript-page-with-python) – Yevhen Kuzmovych Aug 24 '22 at 09:47
  • AJAX uses some URL and you could try to use this URL in your code. It may need use the headers and cookies (like in real browser). Usually AJAX get JSON data so you may no need to convert it. – furas Aug 24 '22 at 10:01
  • your url gives me page with message `Access denied` and `Circus is not available in your region` – furas Aug 24 '22 at 10:02
  • @furas: Do you have any idea where to find the URL? – Stef Renneboog Aug 24 '22 at 10:29
  • you can see all requests in browser using `DevTools` in `Firefox`/`Chrome`, tab: `Network`. And if you set filter: `XHR` then you should see only `AJAX` requests. You can see it for Firefox in my video (without sound) for other page: [How to use DevTools in Firefox to find JSON data in EpicGames.com](https://www.youtube.com/watch?v=TOQz_W2oELU&list=PLV8_1OSqKPKJ-lHXhjX2-NxtUZcQ29c2E) – furas Aug 24 '22 at 10:32
  • I already found the websocket link : wss://wss02.circus.be:443 but now It's unclear how to connect to it.. – Stef Renneboog Aug 24 '22 at 10:40
  • if it uses websocket then it may need Python module [websockets](https://websockets.readthedocs.io/en/stable/) instead of `requests`. And you have to see what browser is sending - and send the same in code. – furas Aug 24 '22 at 10:58

0 Answers0