Trouble Writing CSVs to binary in Pandas

Question

Background

I have some dataframes that I'm trying to split, then write the individual columns into binary files as signed shorts.

Code

for i in range(0, len(filesList)):
    df = pd.read_csv(myPath + '/' + filesList[i], sep='\t', header=None)
    dfA = df[1]
    dfA = dfA - np.mean(dfA)
    dfA = 10000*(dfA/(np.max(dfA)-np.min(dfA)))
    dfA = dfA.astype(int)
    dfA.to_csv(path=writeOutDirectoryPath + '/' + filesList[i] + 'LeadA', mode='wb', index=False)

Everything through making the series dfA into integers (which finishes their conversion into a format that will be readable as signed shorts, I think; all my values are between -6000 and 6000 for this particular post-transform data set.

The Problem

I expect dfA.to_csv to write to the file in binary, hence mode='wb'. I get an error:

f = open(path, mode, errors='replace') ValueError: binary mode doesn't take an errors argument

I must be doing something wrong, but I don't know what. Maybe to_csv can't write binary files out, but I can't find anything in the documentation that would indicate that. Barring telling me what I'm doing wrong with this, what's the best way to convert information in a DataFrame into a form that can be done as binary?

Edit

Writing the file out the CSV is trivial (and I can do that already). If anyone has a method to convert a CSV to a binary file, I'm completely open to that as well.

CSV is a plain text (e.g. ASCII) "comma separated values" file... doing this as a binary makes no sense. — pyj, Feb 12 '16 at 00:14
I know what a CSV is, but there's no explicit to_text or to_binary and the error message is cryptic. If there's something in the documentation I missed, please do feel free to point it out. — khtad, Feb 12 '16 at 00:16

score 0 · Answer 1 · edited May 23 '17 at 12:07

Python 3 is giving an error for the builtin open() function when it is trying to open the file from within pandas, since binary file mode only supports None as an error argument. It'll probably work with "w" mode only from pandas.

Python 3 open() docs doesn't explain it, but it is shown in library code here.

You should look here for a full description of writing types appropriately binary using the python struct package.

Trouble Writing CSVs to binary in Pandas

1 Answers1