How to download a zip file from the raw binary returned by a requests POST endpoint?

Question

I need to hit a POST endpoint and save the response which is a binary data as a zip file and download it to my local directory. I have managed to hit the POST endpoint and retrieve the binary data but I am struggling to convert that or save that as a zip file.

The function download_contract is the POST I use to hit the endpoint where contract is the JSON I'm parsing:

def post(cls, url, access_token, obj=None):
    return WebResponse.from_requests(requests.post(url, data=None if obj is None else obj.json(), headers=cls.secure_header(access_token) if obj is None else cls.secure_json_header(access_token)))

def download_contract(self, contract):
    return SecureWebRequest.post(self.CONTRACT_EXPORT_URL, obj=contract, access_token=self.access_token)

The following code is about me hitting the endpoint and retrieving it's response and trying to save it as a zip file.

@then(u'I will download file from S3')
def step_impl(context):
    print(context.response)
    file_name = 'myzip.zip'
    with context.services.autoimport.download_contract(DownloadContract(context.program_id, complete_filename(context.filename))) as response, open(file_name, 'wb') as out_file:
        shutil.copyfileobj(response, out_file)
        with zipfile.ZipFile(file_name) as zf:
            zf.extractall()

What's returned when I print(context.response) is as follows:

WebResponse(200,m_-Jinstrument.xmlmPÍ Â0
žď)B_ÖQ1CđŕYń8ĘIkGßŢˇ)Ă\ň}|?FłČeqÁťť×=Lť&sRÁĐxÜăcŇĽHżÁ:Źů9 üż7OĚhtÓé`Ä#CŤ3>ÚŇŞVhŕşwZÂč8yłś 32Wű\ep

WebResponse is the (status_code, result) where status_code corresponds to the HTTP Status code of 200 and result is the raw data returned by the endpoint.

class WebResponse():
    STATUS_OK = 200
    STATUS_CREATED = 201
    STATUS_ACCEPTED = 202
    STATUS_NOCONTENT = 204
    STATUS_BADREQUEST = 400
    STATUS_UNAUTHORIZED = 401
    STATUS_FORBIDDEN = 403
    STATUS_NOTFOUND = 404
    STATUS_CONFLICT = 409
    STATUS_INVALID = 422
    STATUS_INTERNALSERVER_ERROR = 500

    def __init__(self, r, result=None):
        self.status_code = r.status_code
        self.text = r.text
        self.headers = r.headers
        self.result = r.text if result is None else result

    def __repr__(self):
        return 'WebResponse(%s, %s)' % (self.status_code, self.result)

Error Thrownback:

Traceback (most recent call last): File "/Users/ss/anaconda/envs/autoimport-server/lib/python3.4/site-packages/behave/model.py", line 1456, in run match.run(runner.context) File "/Users/ss/anaconda/envs/autoimport-server/lib/python3.4/site-packages/behave/model.py", line 1903, in run self.func(context, *args, **kwargs) File "features/steps/miu_connector_steps.py", line 255, in step_impl with context.services.autoimport.download_contract(DownloadContract(context.program_id, complete_filename(context.filename))) as response, open(file_name, 'wb') as out_file: AttributeError: exit

http://stackoverflow.com/a/9419208/6626530 - may be you should to try urllib2.urlopen — Shijo, Jan 13 '17 at 14:32
@Shijo That requires a link with the zip file in it though, I only have a POST endpoint which I can only hit. i.e. "http://accautoimport.miuinsights.com/v1/contractsExport" — user4659009, Jan 13 '17 at 14:40
how about http://stackoverflow.com/questions/13137817/how-to-download-image-using-requests — Shijo, Jan 13 '17 at 14:47

score 0 · Answer 1 · answered Jan 13 '17 at 14:50

0

Your response looks like 'WebResponse(200,m_-Jins ...' and that is what you are writing into a file shutil.copyfileobj(response, out_file).

I assume you want to write response.result into the file. Btw. what is the content in out_file ?

answered Jan 13 '17 at 14:50

Maurice Meyer

17,279
4
30
47

Yes, I want to write `response.result` into the file. Nothing, it fails before it gets to that line – user4659009 Jan 13 '17 at 15:09
Whats the error (edit your post with full traceback) ? – Maurice Meyer Jan 13 '17 at 15:11
I get the following back when I try your suggested solution out. `shutil.copyfileobj(context.response.result, f) File "/Users/ss/anaconda/envs/autoimport-server/lib/python3.4/shutil.py", line 67, in copyfileobj buf = fsrc.read(length) AttributeError: 'str' object has no attribute 'read'` – user4659009 Jan 13 '17 at 16:00
I've added the full traceback to the post above. – user4659009 Jan 13 '17 at 16:04

score 0 · Answer 2 · answered Jan 13 '17 at 17:14

The init below was missing a r.raw where r refers to the requests.

class WebResponse():
    STATUS_OK = 200
    STATUS_CREATED = 201
    STATUS_ACCEPTED = 202
    STATUS_NOCONTENT = 204
    STATUS_BADREQUEST = 400
    STATUS_UNAUTHORIZED = 401
    STATUS_FORBIDDEN = 403
    STATUS_NOTFOUND = 404
    STATUS_CONFLICT = 409
    STATUS_INVALID = 422
    STATUS_INTERNALSERVER_ERROR = 500

    def __init__(self, r, result=None):
        self.status_code = r.status_code
        self.text = r.text
        self.headers = r.headers
        self.raw = r.raw
        self.result = r.text if result is None else result

I then changed my code to the following to get it working:

@then(u'I will download the zip file from the service')
def step_impl(context):
    with open(context.filename, 'wb') as fd:
        for chunk in context.response.iter_content(chunk_size=128):
            fd.write(chunk)

How to download a zip file from the raw binary returned by a requests POST endpoint?

2 Answers2