3

Does python's csv writer not support binary mode anymore?

I haven't had to write in 'b' mode until now and i'm getting very annoying errors like so:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-030fb0c9dc9a> in <module>()
  4 with open('test.csv', 'wb') as f:
  5     w = csv.writer(f)
----> 6     w.writerows(rows)

TypeError: a bytes-like object is required, not 'str'

Code:

import csv

rows = [b'1,2,3', b'4,5,6', b'7,8,9']
with open('test.csv', 'wb') as f:
    w = csv.writer(f)
    w.writerows(rows)

If anyone could explain the error that would be great. I'm passing in an iterable where every element is a byte sequence but I still get an error about the input not being 'bytes' but instead being 'str.' This behavior seems unexpected.

I know the above code snippet can write to a normal file if I turn off the binary mode. If anyone has a solution or suggestion that is constructive I would very much appreciate it.

yvanscher
  • 1,009
  • 1
  • 13
  • 16
  • 2
    "Does python's csv writer not support binary mode anymore?" Correct. The `csv` module in Python 3 needs input & output files to be opened in text mode. – PM 2Ring May 01 '18 at 17:19
  • 5
    Can you _please_ replace those images with actual text? Please see [Why may I not upload images of code on SO when asking a question?](http://meta.stackoverflow.com/questions/285551/why-may-i-not-upload-images-of-code-on-so-when-asking-a-question) – PM 2Ring May 01 '18 at 17:20
  • 1
    @PM2Ring thanks for the comment. i'm well aware of the benefits of adding it as a text block. ill fix is as soon as i have 5 minutes to spare. – yvanscher May 01 '18 at 17:29
  • 1
    Can you clarify why you _must_ have a binary file? Are you writing data that's not compatible with 7 bit ASCII or UTF-8? If so, you may need to use another encoding, eg Latin1. – PM 2Ring May 01 '18 at 17:32
  • @PM2Ring It's an api interface that only supports binary write modes. Not something I have any control over. Stuck with this interface. The answer below by user2357112 works well, especially with the write_through=True flag so it just sends everything to the buffer of the underlying binary file handle and just does the text->binary conversion. – yvanscher May 03 '18 at 13:19

1 Answers1

17

The csv module in Python 3 always attempts to write strings, not bytes:

To make it as easy as possible to interface with modules which implement the DB API, the value None is written as the empty string. [...] All other non-string data are stringified with str() before being written.

That means you have to pass it a file object that accepts strings, which usually means opening it in text mode.

If you are stuck with a file object that wants bytes, wrap it in an io.TextIOWrapper to handle str->bytes encoding:

# assuming you want utf-8
with io.TextIOWrapper(binary_file, encoding='utf-8', newline='') as text_file:
    w = csv.writer(text_file)
user2357112
  • 260,549
  • 28
  • 431
  • 505
  • 1
    binary mode does not take a newline argument – Christopher Pisz Oct 18 '19 at 18:55
  • 1
    @ChristopherPisz: Yeah, but I'm passing the `newline` argument to `io.TextIOWrapper`, not to the call that opens the original file in binary mode. `io.TextIOWrapper` *does* take a newline argument. – user2357112 Oct 18 '19 at 20:16