Streaming a generated CSV with Flask

Question

I have this function for streaming text files:

def txt_response(filename, iterator):
    if not filename.endswith('.txt'):
        filename += '.txt'
    filename = filename.format(date=str(datetime.date.today()).replace(' ', '_'))
    response = Response((_.encode('utf-8')+'\r\n' for _ in iterator), mimetype='text/txt')
    response.headers['Content-Disposition'] = 'attachment; filename={filename}'.format(filename=filename)
    return response

I am working out how to stream a CSV in a similar manner. This page gives an example, but I wish to use the CSV module.

I can use StringIO and create a fresh "file" and CSV writer for each line, but it seems very inefficient. Is there a better way?

score 17 · Accepted Answer · edited Oct 02 '18 at 10:48

According to this answer how do I clear a stringio object? it is quicker to just create a new StringIO object for each line in the file than the method I use below. However if you still don't want to create new StringIO instances you can achieve what you want like this:

import csv
import StringIO

from flask import Response


def iter_csv(data):
    line = StringIO.StringIO()
    writer = csv.writer(line)
    for csv_line in data:
        writer.writerow(csv_line)
        line.seek(0)
        yield line.read()
        line.truncate(0)
        line.seek(0)  # required for Python 3


def csv_response(data):
    response = Response(iter_csv(data), mimetype='text/csv')
    response.headers['Content-Disposition'] = 'attachment; filename=data.csv'
    return response

If you just want to stream back the results as they are created by csv.writer you can create a custom object implementing an interface the writer expects.

import csv

from flask import Response


class Line(object):
    def __init__(self):
        self._line = None
    def write(self, line):
        self._line = line
    def read(self):
        return self._line


def iter_csv(data):
    line = Line()
    writer = csv.writer(line)
    for csv_line in data:
        writer.writerow(csv_line)
        yield line.read()


def csv_response(data):
    response = Response(iter_csv(data), mimetype='text/csv')
    response.headers['Content-Disposition'] = 'attachment; filename=data.csv'
    return response

Great answer. I also added a `writer.writeheader()` and `yield line.read()`before the for loop to add the headers — theannouncer, Jan 14 '20 at 20:17

score 3 · Answer 2 · answered Nov 02 '21 at 11:38

A slight improvement to Justin's existing great answer. You can take advantage of the fact that csv.writerow() returns the value returned by the underlying file's write call.

import csv
from flask import Response

class DummyWriter:
    def write(self, line):
        return line

def iter_csv(data):
    writer = csv.writer(DummyWriter())
    for row in data:
        yield writer.writerow(row)

def csv_response(data):
    response = Response(iter_csv(data), mimetype='text/csv')
    response.headers['Content-Disposition'] = 'attachment; filename=data.csv'
    return response

score 1 · Answer 3 · answered Mar 19 '20 at 22:31

If you are dealing with large amounts of data that you don't want to store in memory then you could use SpooledTemporaryFile. This would use StringIO until it reaches a max_size after that it will roll over to disk.

However, I would stick with the recommended answer if you just want to stream back the results as they are created.

Streaming a generated CSV with Flask

3 Answers3