1

I am using Pythong requests lib to retrieve data from New Zealand Statistics website: http://www.stats.govt.nz/infoshare/ExportDirect.aspx. The http POST request is in format of "multipart/form-data". I googled and found that requests is a great library that supports multipart post. However, my code didn't get the result that should be retrieved from the server. Instead, It only gives me the source codes of website http://www.stats.govt.nz/infoshare/ExportDirect.aspx.

Can some one help me with this problem?

from urllib.request import urlopen
from bs4 import BeautifulSoup
import requests

r = requests.get('http://www.stats.govt.nz/infoshare/ExportDirect.aspx')


soup = BeautifulSoup(r.text)
view_state = soup.find("input",id="__VIEWSTATE")["value"]
event_validation = soup.find("input",id="__EVENTVALIDATION")["value"]

post_data = {"__EVENTTARGET": "",
             "__EVENTARGUMENT": "",
             "__VIEWSTATE": view_state,
             "__EVENTVALIDATION": event_validation,
             "ctl00$MainContent$tbMissingText": "..",
             "ctl00$MainContent$TimeVariableSelector$tbSelected": "1",
             "ctl00$MainContent$TimeVariableSelector$lbVariableOptions": "0",
             "ctl00$MainContent$rblCSVFormat": "columns",
             "ctl00$MainContent$rblSeriesDescription": "no_description",
             "ctl00$MainContent$fuSearchFile": (open('Alcohol.sch'), 'application/octet-stream')
    }


r1 = requests.post('http://www.stats.govt.nz/infoshare/ExportDirect.aspx?AspxAutoDetectCookieSupport=1', files = post_data)


print(r1.text)

input('Enter to exit...')
ECONVBA
  • 41
  • 1
  • 3
  • 2
    why do you do `soup = BeautifulSoup(urlopen('http://www.stats.govt.nz/infoshare/ExportDirect.aspx'))`? `soup = BeautifulSoup(r.text)` will be enough here. – dexter Aug 12 '14 at 08:14
  • You can try the solution of [The answer.](http://stackoverflow.com/a/15900453/1113211). – salmanwahed Aug 12 '14 at 09:11
  • And also create the `soup` like this, `soup = BeautifulSoup(r.text)` as you already have a `response` object `r` – salmanwahed Aug 12 '14 at 09:12
  • 1
    Please, add example of file 'Alcohol.sch'. – NorthCat Aug 12 '14 at 15:20
  • The 'Alcohol.sch' file contains two IDs that are used to send requests to the server. The codes are: SEPQ.SBC3A SEPQ.SBC3B – ECONVBA Aug 15 '14 at 03:25

0 Answers0