-1

I'm using Python \ Selenium \ Chrome web drivers to perform web scraping in Visual Studio Code.

If I send a GET i.e.:

driver.get('https://my_test_website/customerRest/show/?id=123')

How do I actually view the contents of the reply? If I run the URL in Chrome, I can view all the data in one huge body:

enter image description here

How do I view it in Python? I ultimately want to insert it into a SQL table but the 1st step is to view via Python.

Any help would be greatly appreciated.

Michael
  • 2,507
  • 8
  • 35
  • 71

1 Answers1

1

To view the contents of the <body> tag you need to use the page_source property.

page_source: Gets the source of the current page.

Usage:

driver.page_source

To print the contents within the console:

print(driver.page_source)
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • So the body contains data like this: "auditCreatedBy":"admin","auditCreatedByDisplayName":"admin","auditUpdateDate":"2021-08-10T15:52:16Z","auditUpdatedBy":"CorePullData" How could I retrieve something specific like the 'auditCreatedBy'? – Michael Mar 08 '22 at 13:29
  • @Michael that would depend on the type of element which contains the desired information. You can easily [scrape](https://stackoverflow.com/a/57688853/7429447) it using [Selenium](https://stackoverflow.com/a/54482491/7429447) and/or [BeautifulSoup](https://stackoverflow.com/a/47871704/7429447) – undetected Selenium Mar 08 '22 at 13:33
  • I've tried find element by ID and by name but it says it can't find them? – Michael Mar 08 '22 at 14:10
  • 1
    @Michael Shouldn't that be a seperate question all together? Feel free to raise a new question as per your new requirement. – undetected Selenium Mar 08 '22 at 14:11