I am scraping an Applicant Tracking System (BrassRing). I can get logged-in without an issue using Selenium and get the webpage I'm interested in scraping. As I was searching for the table I discovered that the data I want is stored in a jsonGrid
.
Everything I can find about Selenium and scraping does not cover how to scrape the contents of a JSON grid.
There are 8 columns in this grid/table that have all of the date beneath them (some cells are empty and that's ok).
As far as I can tell the columns are labeled as follows in the JSON itself though the website displays them slightly differently:
Action Type
Action Date
Action By
Details
Name
emailfrom
emailto
folderid
Here is the first part of the website code that shows the headers and some of the column values.
It would be great if you could provide some information on how I can just scrape the JSON grid/JSON data from the site.
<input type="hidden" name="Grid$jsonData183" id="Grid_jsonData183" class="jsonGridData" value="[{"ActionType": "Communication - Email","ActionDate": "18-Oct-2019 14:14:25","ActionBy": "Manager, Automation ()","Details": "Status: Sent as to","Name": "<a href=\"#\"/ class=\"ViewCommunication\">Not Selected</a>","emailfrom": "Manager, Automation ()","emailto": "Smith, John(john.smith@notreal.com)","hm_category": "5","hm_Folderid": "6537489","hm_ReqId": "-1","hm_content": "1","hm_md_communication_type": "Communication - Email","hm_md_communication_correspondenceid": "1","hm_md_communication_correspondenceresumeid": "46878397","hm_pushportal": "0","hm_unpostportal": "0","hm_postportall": "0","hm_PortalExpired": "0","hm_md_communication_agencycodetypeid": "0","hm_md_communication_agencycodeid": "0","hm_md_communication_userid": "41","hm_md_RecipientType": "4","hm_EmailLogId": "0","hm_md_ReceiverUserID": "0","hm_md_fid": "6537489","hm_md_rid": "6454343","hm_md_rftid": "17","hm_md_rsts": "0","hm_md_myfolder": "0","foldername": "<a href='#' class='ViewFolder'>1738995BR:Customer Service Associate II</a>","hm_md_afl": "0","hm_md_rfl": "1","hm_md_rlg": "en", "rowmetadata": "<div><div name=\"category\" value=\"5\"></div><div name=\"folderid\" value=\"6537489\"></div><div name=\"reqid\" value=\"-1\"></div><div name=\"content\" value=\"1\"></div><div name=\"md_communication_type\" value=\"Communication+-+Email\"></div><div name=\"md_communication_correspondenceid\" value=\"1\"></div><div name=\"md_communication_correspondenceresumeid\" value=\"46878397\"></div><div name=\"pushportal\" value=\"0\"></div><div name=\"unpostportal\" value=\"0\"></div><div name=\"postportall\" value=\"0\"></div><div name=\"portalexpired\" value=\"0\"></div><div name=\"md_communication_agencycodetypeid\" value=\"0\"></div><div name=\"md_communication_agencycodeid\" value=\"0\"></div><div name=\"md_communication_userid\" value=\"41\"></div><div name=\"md_recipienttype\" value=\"4\"></div><div name=\"emaillogid\" value=\"0\"></div><div name=\"md_receiveruserid\" value=\"0\"></div><div name=\"md_fid\" value=\"6537489\"></div><div name=\"md_rid\" value=\"6454343\"></div><div name=\"md_rftid\" value=\"17\"></div><div name=\"md_rsts\" value=\"0\"></div><div name=\"md_myfolder\" value=\"0\"></div><div name=\"md_afl\" value=\"0\"></div><div name=\"md_rfl\" value=\"1\"></div><div name=\"md_rlg\" value=\"en\"></div></div>"},{"ActionType": "Communication - Email","ActionDate": "18-Oct-2019 13:24:13","ActionBy": "Manager, Automation ()","Details": "Status: Sent as to","Name": "<a href=\"#\"/ class=\"ViewCommunication\">Not Selected</a>","emailfrom": "Manager, Automation ()","emailto": "Smith, John(john.smith@notreal.com)","hm_category": "5","hm_Folderid": "6513663","hm_ReqId": "-1","hm_content": "1","hm_md_communication_type": "Communication - Email","hm_md_communication_correspondenceid":