I am using Pythong requests lib to retrieve data from New Zealand Statistics website: http://www.stats.govt.nz/infoshare/ExportDirect.aspx. The http POST request is in format of "multipart/form-data". I googled and found that requests is a great library that supports multipart post. However, my code didn't get the result that should be retrieved from the server. Instead, It only gives me the source codes of website http://www.stats.govt.nz/infoshare/ExportDirect.aspx.
Can some one help me with this problem?
from urllib.request import urlopen
from bs4 import BeautifulSoup
import requests
r = requests.get('http://www.stats.govt.nz/infoshare/ExportDirect.aspx')
soup = BeautifulSoup(r.text)
view_state = soup.find("input",id="__VIEWSTATE")["value"]
event_validation = soup.find("input",id="__EVENTVALIDATION")["value"]
post_data = {"__EVENTTARGET": "",
"__EVENTARGUMENT": "",
"__VIEWSTATE": view_state,
"__EVENTVALIDATION": event_validation,
"ctl00$MainContent$tbMissingText": "..",
"ctl00$MainContent$TimeVariableSelector$tbSelected": "1",
"ctl00$MainContent$TimeVariableSelector$lbVariableOptions": "0",
"ctl00$MainContent$rblCSVFormat": "columns",
"ctl00$MainContent$rblSeriesDescription": "no_description",
"ctl00$MainContent$fuSearchFile": (open('Alcohol.sch'), 'application/octet-stream')
}
r1 = requests.post('http://www.stats.govt.nz/infoshare/ExportDirect.aspx?AspxAutoDetectCookieSupport=1', files = post_data)
print(r1.text)
input('Enter to exit...')