Downloading multiple csv files from Google App Engine

Question

So I found a lot of good info on here from how to download data from GAE to a csv.. where I'm stuck is how to export multiple csv files

Summary: I have a webapp that takes multiple inputs and for each input, it sends data to the datastore and then writes from the datastore to the csv. Sample code is below:

class RunScript(webapp2.RequestHandler):
def post(self): 
    listOfTickers = self.request.get('Stocks').split(", ")
    for i in listOfTickers:
        self.main(cgi.escape(i))

main method calls other functions to get the data and ends up writing to csv via the code below:

item_query=Item.query().order(Item.keyOfItem).fetch()
    self.response.headers['Content-Type'] = 'text/csv'
    self.response.headers['Content-Disposition'] = 'attachment; filename=file.csv'
    writer = csv.writer(self.response.out)
    for a in item_query:
        rowToWrite=a.row+a.value
        writer.writerow(rowToWrite)
        #empty datastore as data is drawn out
        a.key.delete()

at this point, the sub function ends and we go back to the for loop in the post() function. I thought the sub function ending would trigger GAE to download the first csv and then download the second csv after it goes through the code again. However, what happens is I get one csv at the end with both sets of data in it.

Any thoughts as to why this is happening? Any help is appreciated!

score 1 · Accepted Answer · answered Apr 10 '14 at 05:17

1

You keep writing data to the same response, i.e. to the same file. A response cannot return multiple files.

You should create several files, archive them together into a single file, and then return this archive.

answered Apr 10 '14 at 05:17

Andrei Volgin

40,755
6
49
58

hmm.. I'm still learning the nuances of GAE so please excuse me if this is basic but can you explain that a little bit more? I assume you mean something like create multiple .csvs and then zip them up and download that one zip file; how would I do that? or is there a different way to call multiple responses? – UberBballPlayer30 Apr 10 '14 at 05:40
Each request has only one response. There are no ways to have multiple responses without multiple requests. This is not specific to GAE. And yes, you need to create a zip from several csv files. This should be a separate question - or there may be one already asked and answered. – Andrei Volgin Apr 10 '14 at 05:43

score 0 · Answer 2 · edited May 23 '17 at 12:00

To create a response that has multiple separate pieces of data in it, you should look into multipart content type. This is usually used in request to send multipart form data, but it can also be used in response.

There is some support in browsers for multipart response: Browser support of multipart responses

There is also a mpAjax javascript library that can help you parse the multipart response. You might want to test it.

score 0 · Answer 3 · answered Apr 10 '14 at 09:07

As an alternative yo can also write the cvs's to a single zip archive.

Or you can write you csv files to cloudstorage and use cloudstorage to download the files if you need them.

See this gist for using cloudstorage. It contains an example to create a zip archive.

Downloading multiple csv files from Google App Engine

3 Answers3