Parse Beautifulsoup output to csv file for download

Question

I've gone through all the questions on this site and googled but really struggling to take a Beautifulsoup output (which appear to be lists) and return it in csv format so that the user of my Flask app can download in the browser. Here's the code:

html = col_result.__html__()
bs = BeautifulSoup(html)
table = bs.find(lambda tag: tag.name == 'table')
headers = table.findAll(lambda tag: tag.name == 'th')
rows = table.findAll(lambda tag: tag.name == 'tr')

with open('export_file.csv', 'w+', newline='') as f:
    file = csv.writer(f)
    file.writerow(headers)
    file.writerows(rows)
rfile = csv.reader(open('export_file.csv', newline=''))

return Response(
    rfile,
    mimetype="text/csv",
    headers={"Content-disposition":
             "attachment; filename=export_file.csv"})

The csv file is blank when downloaded. I have imported csv & bs4 modules. Can anyone advise how to produce the raw csv data from the html so that it can be passed into the 'rfile' variable in my return response() section of the code as I cannot get this to work?

For example if I iterate through the csv.reader object and print each line I get,

['Firstname', 'Surname', 'Department', 'Manager', 'Absence Periods', 'Late Instances'] ['Renata', 'Krzysik', 'Gifts', 'Michael de Jäger', '0 Absence', '0 Lateness']...

...but I cant work out how to parse to csv format (without the lists and html tags)and assign it to the rfile variable?

you can just use the `csv.writer()` to write that data in the file. — Ma0, Sep 07 '16 at 11:24
have you tried returning the content of the `csv` file instead of returning the `file` itself as an attachment? — Saeid, Sep 07 '16 at 11:27
Why are you not just using `table.find_all( 'th')` etc.. ? What is the point of the lambda? — Padraic Cunningham, Sep 07 '16 at 12:25

Saeid · Accepted Answer · 2016-09-07T11:53:47.973

The first parameter of the Response class should be

response – a string or response iterable

So I don't think you can pass a csv.reader as this parameter. Try to pass the string content of the file instead.

Or you can use make_response and pass the contents of csv file to it:

from flask import make_response

html = col_result.__html__()
bs = BeautifulSoup(html)
table = bs.find(lambda tag: tag.name == 'table')
headers = table.findAll(lambda tag: tag.name == 'th')
rows = table.findAll(lambda tag: tag.name == 'tr')

with open('export_file.csv', 'w+', newline='') as f:
    file = csv.writer(f)
    file.writerow(headers)
    file.writerows(rows)

with open('export_file.csv', 'r') as csv:
    response = make_response(csv.read())
    response.headers["Content-Disposition"] = "attachment; filename=books.csv"
    return response

Thank you, make-response was exactly what I needed! – MattE Sep 07 '16 at 11:59 — MattE, Sep 07 '16 at 11:59

Parse Beautifulsoup output to csv file for download

1 Answers1