-1

I'm trying to scrape the current rank of a player from lolprofile.net, but it's never up to date, so I somehow have to hit the "update" button before getting my data. I've tried scraping the 'updated' page the button leads to, without getting different results. Suspected the site might take some time to refresh the data and therefore added a timer, nothing happened differently. My code:

from bs4 import BeautifulSoup import requests import time

url = ['https://lolprofile.net/de/summoner/euw/Sidestep%20Sabrina#update', 'https://lolprofile.net/de/summoner/euw/Diadormus#update', 'https://lolprofile.net/de/summoner/euw/DeluxeFuture#update', 'https://lolprofile.net/de/summoner/euw/Natron#update', 'https://lolprofile.net/de/summoner/euw/51l3nc3#update']

for x in url:

html_text = requests.get(x).text
time.sleep(5)
soup = BeautifulSoup(html_text, 'lxml')

player_name = soup.find('h1').text
rank = soup.find('span', class_ = 'tier').text
LP = soup.find('span', class_ = 'lp').text

print(f'{player_name} ist {rank} mit {LP}')

input()

How should I approach this problem? There's been a similar question asked here but it never got answered.

Diadormus
  • 1

1 Answers1

0

If you use the browser development tools (Network tab) you can see that the "update" button sends an AJAX request to the server to a different address to the one that shows if you hover over it.

Screnshot of development tools showing request

Therefore you need to be using that address to get the updated details. In this case the correct address is: https://lolprofile.net/index.php?page=summoner&ajaxr=1&region=euw&name=Sidestep%20Sabrina

Ari Cooper-Davis
  • 3,374
  • 3
  • 26
  • 43
  • it doesn't seem to be updated either. e.g. compare https://lolprofile.net/index.php?page=summoner&ajaxr=1&region=euw&name=DeluxeFuture and https://www.leagueofgraphs.com/summoner/euw/DeluxeFuture -- the 2nd link is up to date – Diadormus Feb 19 '22 at 11:03
  • Those are two entirely different websites (`lolprofile` and `leagueofgraphs`). The first link matches the data shown on [the associated updated page on `lolprofile`](https://lolprofile.net/de/summoner/euw/DeluxeFuture#update) as expected. – Ari Cooper-Davis Feb 19 '22 at 15:57
  • yes, I'm aware it's different websites, I used it as comparison. the data on leagueofgraphs is up to date, while the url you posted (https://lolprofile.net/index.php?) still contains older data – Diadormus Feb 20 '22 at 13:21
  • The link I posted contains the same data as the url in your question: [mine](https://lolprofile.net/index.php?page=summoner&ajaxr=1&region=euw&name=Sidestep%20Sabrina) and [yours](https://lolprofile.net/de/summoner/euw/Sidestep%20Sabrina). If you want to webscrape `leagueofgraphs` instead then you should probably edit your question, or ask a new one altogether. – Ari Cooper-Davis Feb 20 '22 at 14:01