1

I currently log on to site using POST, I then use POST to access a data-table. When I add a form data payload to the POST accessing the data-table of interest (to change the sort by method of the table) the html returned is for the unsorted table.

referrer = "https://edx.standardandpoors.com/mailbox/jsp/MBIList.jsp"

with requests.Session() as s:
    p = s.post(spURL_post, data=spPayload, proxies=proxyDict, verify = True)
    q= s.post(referrer, data=dataSort, proxies = proxyDict, verify = True)
    dataPageResponse = s.get(referrer, proxies = proxyDict, verify = True)

html = BeautifulSoup(dataPageResponse.content,"lxml")

p: I log in, q: I access the data table on the following page (hopefully sorted)

I have tried verify=True, using POST with the data payload after the first POST to get to the data table, sending the data payload using json.dumps(data_payload_dict), as a string, and as the dictionary shown below and I still cannot get the correct (sorted table) HTML.

The sorting data payload that I am submitting is pulled directly from the URL encoded form data shown in the Network Inspector after manually clicking the sort button:

dataSort = {"MailboxPath": "All", "MessageNamePattern": "**",
            "MsgNamePattern": "", "MessageId": "", "StartDate": "2018-06-03",
            "StartTime": "11%3A57", "StartAMPM": "AM", "EndDate": "2018-07-03",
            "EndTime": "11%3A58", "EndAMPM": "AM", "id": "", "OrderBy": "CreatedDateTime",
            "lastSortOrder": "CreatedDateTime","ascending": "Yes",
            "itemsPerPage": "100","currentPage": "1",
            "persistOrder": "Yes"}

Please checkout the form data per the inspector: Form Data per Inspector The only thing I can think of is that I am not submitting the payload data in a format the website is expecting. Here is the top of the Request Headers portion Request Headers. Perhaps I need to encode the form data payload?

Thanks

Community
  • 1
  • 1
user129818
  • 15
  • 6

0 Answers0