0

I am trying to write a program that automatically clicks on certain places based on what is displayed. The website I am trying to do this on is Gimkit. I am using a python web-scraper and auto-clicker to perform this task. The problem I am running into is that the player side URL is the same, no matter what html is loaded. This leads to the scraper only getting the html for the first page, which is the game PIN page. This means that my scraper is getting no useful information and is simply stuck. When I fill out the information on my device and check the URL, it is the same. There is no difference in the URL when I am entering the PIN or playing the game, but when I inspect element, it obviously is different html. Right now, I am using Requests, BeautifulSoup4, and lxml to get and format the html from the site. How can I access the html for the gameplay page instead of the PIN page?

Related: How can I keep the same url in the address bar for every page?

Jackson148
  • 115
  • 1
  • 2
  • 9
  • 1
    Web apps or POST forms will change the session state but not the URL. You need a library like this: https://pypi.org/project/pyppeteer/ to simulate form submissions and navigation clicks. –  Feb 12 '20 at 00:16
  • 1
    _Right now, I am using BeautifulSoup4 and lxml to get the html from the site._ How are you getting the HTML in the first place, though? Using Requests? – AMC Feb 12 '20 at 01:51
  • @AMC yes sorry, I forgot to include that, I will add that now. – Jackson148 Feb 12 '20 at 12:32
  • @ChrisG Thanks for the help, i was able to navigate to the proper page, but I am now struggling to scrape the html. Do you know if there is a way to do this using pyppeteer, or would you suggest something else? Thank you again! – Jackson148 Mar 06 '20 at 21:16

0 Answers0