Stack Overflow
Stack Overflow

Questions tagged [csvkit]

Questions related to use of the cvskit utilities suite, for converting to and working with CSV format files. Check the cvskit Issues list before posting your question, in case there's a known work-around or solution.

The cvskit suite provides script-ready command-line utilities to work with the ubiquitous file format.

Supported platforms

As documented, csvkit supports the following versions of Python. Where appropriate, include these tags in your question, as well as any relevant shell tag:

It is tested on , but has been used successfully by others on both and .

References

54 questions
10
votes
4 answers

How to let csvkit/csvsql generate insert statements for csv file?

I want to generate sql insert statements for some csv file. I can do this using a one-liner awk script such as: awk -F "\t" '{printf("INSERT INTO T_COMMON_ENUM_VALUE (id,name,category_id) values (%s, '\''%s'\'', %s, %s);\n", $1, $2, $3, $4)}' But…
Mert Nuhoglu
  • 9,695
  • 16
  • 79
  • 117
6
votes
2 answers

in2csv: iter_rows() got an unexpected keyword argument 'row_offset'

I'm using in2csv from csvkit version 1.0.3, installed using pip with Python 3.7. When using the tool for the most basic conversion task i.e. in2csv filename.xlsx > test.csv I was hit with error iter_rows() got an unexpected keyword argument…
idazuwaika
  • 2,749
  • 7
  • 38
  • 46
2
votes
1 answer

csvcut: error: unrecognized arguments: filename.csv

Trying to reorder columns in a csv. This works (calling single column names): $ csvcut -c "Node ID" amazon_categories_nodes.csv and this works (calling multiple column indexes): $ csvcut -c 4,1,3,5 amazon_categories_nodes.csv But this doesn't work…
AK91
  • 671
  • 2
  • 13
  • 35
2
votes
2 answers

csvkit in2csv - how to convert a single json object to two-column csv

Looking for a one liner with csvkit. From a plain json object { "whatever": 2342, "otherwise": 119, "and": 1, "so": 2, "on": 3 } Want this csv whatever,2342 otherwise,119 and,1 so,2 on,3 I basically want this command to work, but it…
Jason Dunkelberger
  • 1,207
  • 14
  • 18
2
votes
0 answers

csvjoin error: "coercing to Unicode: need string or buffer, LazyFile found"

I'm trying to join two files using the csvjoin command from csvkit. One csv is much larger than the other, but they both have a column in common, an RD code (with a column name of "RD"). Assuming that it's not a problem for csvjoin to match on just…
impostorsyndrome
  • 103
  • 1
  • 10
2
votes
1 answer

Csvkit : use "in2csv" in python code

I'd like to use in2csv to convert excel files into csv in my python code, but without running it as a separate process using exec. Instead, I'd like to directly import the function and use it, like in2csv("input.xls", "output.csv"), or still better,…
tobiasBora
  • 1,542
  • 14
  • 23
2
votes
3 answers

piping from shell into jupyter notebook cell

Does anyone know how to stream the output of a shell command (a chain of csvkit tool invocations) into a jupyter notebook cell, but specifically into a Pandas DataFrame. From the cell's content it would look something like this: output = !find…
Dmitry B.
  • 9,107
  • 3
  • 43
  • 64
2
votes
1 answer

csvkit: for csv to Table, how do you preserve quoted strings when

When using csvkit, I'm having trouble keeping character data from getting transformed to numeric data. For the example below, my first column gets transformed into an 'int' Data:…
rprasad
  • 366
  • 3
  • 8
2
votes
2 answers

How to move column or swap two columns with csvkit?

I'm using csvkit to manipulate csv files, but can't find how to do this: move one column before or after another column swap two columns but keep others unchanged Does someone know how to do it?
Freewind
  • 193,756
  • 157
  • 432
  • 708
1
vote
1 answer

csvkit/in2csv: AttributeError: 'ReadOnlyWorksheet' object has no attribute 'defined_names'

I have installed csvkit with Homebrew, but when running in2csv foo.xlsx > foo.csv, I'm always getting this error message: AttributeError: 'ReadOnlyWorksheet' object has no attribute 'defined_names' How can I fix it ?
DevonDahon
  • 7,460
  • 6
  • 69
  • 114
1
vote
0 answers

csvcut ignores -e command line option thru pipe

EDIT: problem/issue/solution is explained here: https://github.com/wireservice/csvkit/issues/898 A fix involves setting the environment variable PYTHONIOENCODING Today I learned I need to use the -e ENCODING option to deal my input data (it's ascii…
Ed Beighe
  • 11
  • 3
1
vote
2 answers

Python: lightweight package install, without pip?

I'm packaging up a minimal Ubuntu distro to fit in a 4GB disk image, for use on a VPS. This image is a (C++) webapp which (among other things) writes and runs simple Python scripts to handle conversions between csv and xls files, with csvkit and…
QF0
  • 329
  • 2
  • 14
1
vote
0 answers

csvsql into mysql: uses Timestamp, need Datetime

I am trying to use csvkit / csvsql to import csv into mysql. csvkit detects the datetime columns correctly. It tries to create the table with the type TIMESTAMP, but I need DATETIME. Is there any way to force using DATETIME instead of TIMESTAMP? I…
Petr Blahos
  • 2,253
  • 1
  • 11
  • 14
1
vote
1 answer

how diagnose csvsql "column not found" error

Using csvkit on macOS 10.14.3 input file: head -n 3 Nov_git_v2.csv Model_var,dummy Age, Basket_Size_FP_24, Basket_Size_FP_6, csvsql --query "select Model_var from Nov_gift_v2" Nov_gift_v2.csv produces error: sqlite3.OperationalError) no such…
PatrickR
  • 11
  • 2
1
vote
1 answer

csvkit mysql import fails with VARCHAR requires a length on dialect mysql

After installing csvkit with the following command $ sudo -HE pip install --upgrade -e git+git://github.com/wireservice/csvkit.git@master#egg=csvkit and trying to import a .csv as follows: csvsql --db mysql://root:root@127.0.0.1:3306/jira_test…
pkaramol
  • 16,451
  • 43
  • 149
  • 324
1
2 3 4