1

I am creating a project involving web scraping and web automation. I would like to first submit this form (http://rgsntl.rgs.cuhk.edu.hk/rws_prd_applx2/Public/tt_dsp_timetable.aspx) then once you submit this form, I want to scrape the HTML page that comes up. The problem is I am not sure how to submit this form through a Go program.

I was previously experimenting with Selenium to emulate a web browser but now I think there may be an easier way. I think that I should be able to make a POST request to the same address that the "submit" button of this form makes to and directly use the HTML page that is returned. The problem is that I cannot figure out how to get the address that the submit button makes a POST request to. I would like to ask if there is a way to monitor the address that the button makes a POST request to when it is clicked? Also if you see any flaws with my idea please do let me know. Thank you.

Archit Kithania
  • 177
  • 4
  • 13

1 Answers1

4

Right click mouse and select inspect option. After that select the Network tab. When you fill all the entries and click submit button many urls flashes. Select the top url and under the headers tab you will see the request url for POST method.

preview

See the image

Community
  • 1
  • 1
ashishpm
  • 408
  • 1
  • 6
  • 19