How do I write data into CSV format as string (not file)?

Question

I want to cast data like [1,2,'a','He said "what do you mean?"'] to a CSV-formatted string.

Normally one would use csv.writer() for this, because it handles all the crazy edge cases (comma escaping, quote mark escaping, CSV dialects, etc.) The catch is that csv.writer() expects to output to a file object, not to a string.

My current solution is this somewhat hacky function:

def CSV_String_Writeline(data):
    class Dummy_Writer:
        def write(self,instring):
            self.outstring = instring.strip("\r\n")
    dw = Dummy_Writer()
    csv_w = csv.writer( dw )
    csv_w.writerow(data)
    return dw.outstring

Can anyone give a more elegant solution that still handles the edge cases well?

Edit: Here's how I ended up doing it:

def csv2string(data):
    si = StringIO.StringIO()
    cw = csv.writer(si)
    cw.writerow(data)
    return si.getvalue().strip('\r\n')

Since "\r" and "\n" are considered whitespace, you can actually just use "strip" with no argument: `return si.getvalue().strip()`--unless for some reason you need to preserve spaces at the end. — MTKnife, Aug 25 '20 at 14:43

Tim Pietzcker · Answer 1 · 2012-10-29T11:07:57.867

248

In Python 3:

>>> import io
>>> import csv
>>> output = io.StringIO()
>>> csvdata = [1,2,'a','He said "what do you mean?"',"Whoa!\nNewlines!"]
>>> writer = csv.writer(output, quoting=csv.QUOTE_NONNUMERIC)
>>> writer.writerow(csvdata)
59
>>> output.getvalue()
'1,2,"a","He said ""what do you mean?""","Whoa!\nNewlines!"\r\n'

Some details need to be changed a bit for Python 2:

>>> output = io.BytesIO()
>>> writer = csv.writer(output)
>>> writer.writerow(csvdata)
57L
>>> output.getvalue()
'1,2,a,"He said ""what do you mean?""","Whoa!\nNewlines!"\r\n'

edited Oct 29 '12 at 11:07

answered Feb 06 '12 at 08:27

Tim Pietzcker

328,213
58
503
561

Good example. :) As a sidenote, what is the usual behaviour when encountering newlines inside a CSV file? Is `\n` ok to have in the middle of data, but `\r\n` indicates the end of a record no matter where it appears? (Assuming you're on a platform that uses `\r\n` as the line terminator.) – Li-aung Yip Feb 06 '12 at 08:35
I'm not sure. I would have expected `\r\n` myself because the default parameter for `io.StringIO()` is `newline=None` which enables universal newlines mode, which should automatically use the correct newline style according to your platform. I'm kind of puzzled by this behaviour myself. – Tim Pietzcker Feb 06 '12 at 08:46
@Li-aungYip: I've opened a [question about this](http://stackoverflow.com/questions/9157623/unexpected-behavior-of-universal-newline-mode-with-stringio-and-csv-modules). – Tim Pietzcker Feb 06 '12 at 08:55
Maintaining the `\n` as part of the field data seems like a sane thing to do (because it could legitimately appear as part of text data.) – Li-aung Yip Feb 07 '12 at 02:54
2

Should be `output = StringIO.StringIO()`, `io.StringIO()` will raise TypeError: string argument expected, got 'str'. – Marboni Oct 29 '12 at 10:17
2

@Marboni: StringIO is gone in Python 3 (which is what my solution is written in), and I can't reproduce that error in Python 2.7.3 - although I do get a TypeError in the `writer.writerow(...)` line (`unicode argument expected, got 'str'`). Will look into this. – Tim Pietzcker Oct 29 '12 at 10:24
1

@Marboni: Thanks for the heads-up: I [found the problem](http://stackoverflow.com/questions/13120127/how-can-i-use-io-stringio-with-the-csv-module) with the help of StackOverflow. In Python 2, you need `io.BytesIO()` instead of `io.StringIO()`. – Tim Pietzcker Oct 29 '12 at 11:09
1

@Marboni: In Python 2.7.9 it works with StringIO.StringIO() or io.BytesIO(). – srock Jan 30 '15 at 17:55
Curious: why do you have the `quoting=csv.QUOTE_NONNUMERIC` parameter? – flow2k Mar 08 '19 at 19:52
doesn't work on python3.6: `TypeError: a bytes-like object is required, not 'str'` – nz_21 Jul 25 '19 at 13:02
1

@nz_21: I just re-tested it - works perfectly fine. The second example (written for Python 2, as explicitly stated) will throw that error, of course. – Tim Pietzcker Jul 25 '19 at 13:52

score 73 · Accepted Answer · edited Jan 13 '17 at 21:13

73

You could use StringIO instead of your own Dummy_Writer:

This module implements a file-like class, StringIO, that reads and writes a string buffer (also known as memory files).

There is also cStringIO, which is a faster version of the StringIO class.

edited Jan 13 '17 at 21:13

TH22

1,931
2
17
23

answered Feb 06 '12 at 08:23

NPE

486,780
108
951
1,012

3

cStringIO was gone away since python 3.0. Please use `io.StringIO` instead – Pei Apr 06 '21 at 01:18

score 8 · Answer 3 · answered Apr 21 '20 at 12:11

since i use this quite a lot to stream results asynchronously from sanic back to the user as csv data i wrote the following snippet for Python 3.

The snippet lets you reuse the same StringIo buffer over and over again.


import csv
from io import StringIO


class ArgsToCsv:
    def __init__(self, seperator=","):
        self.seperator = seperator
        self.buffer = StringIO()
        self.writer = csv.writer(self.buffer)

    def stringify(self, *args):
        self.writer.writerow(args)
        value = self.buffer.getvalue().strip("\r\n")
        self.buffer.seek(0)
        self.buffer.truncate(0)
        return value + "\n"

example:

csv_formatter = ArgsToCsv()

output += csv_formatter.stringify(
    10,
    """
    lol i have some pretty
    "freaky"
    strings right here \' yo!
    """,
    [10, 20, 30],
)

Check out further usage at the github gist: source and test

score 6 · Answer 4 · answered Oct 28 '15 at 15:44

I found the answers, all in all, a bit confusing. For Python 2, this usage worked for me:

import csv, io

def csv2string(data):
    si = io.BytesIO()
    cw = csv.writer(si)
    cw.writerow(data)
    return si.getvalue().strip('\r\n')

data=[1,2,'a','He said "what do you mean?"']
print csv2string(data)

score -1 · Answer 5 · answered May 05 '15 at 19:56

Here's the version that works for utf-8. csvline2string for just one line, without linebreaks at the end, csv2string for many lines, with linebreaks:

import csv, io

def csvline2string(one_line_of_data):
    si = BytesIO.StringIO()
    cw = csv.writer(si)
    cw.writerow(one_line_of_data)
    return si.getvalue().strip('\r\n')

def csv2string(data):
    si = BytesIO.StringIO()
    cw = csv.writer(si)
    for one_line_of_data in data:
        cw.writerow(one_line_of_data)
    return si.getvalue()

Humberto Arocha · Answer 6 · 2015-06-10T03:24:31.907

-2

import csv
from StringIO import StringIO
with open('file.csv') as file:
    file = file.read()

stream = StringIO(file)

csv_file = csv.DictReader(stream)

edited Jun 10 '15 at 03:24

answered Jun 09 '15 at 02:00

Humberto Arocha

17
1

4

Code-only answers are discouraged, you should add some clarification to your answer – Raniz Jun 10 '15 at 04:23

How do I write data into CSV format as string (not file)?

6 Answers6

Linked

Related