Add new column to each row of a .csv in ruby

Question

I'm attempting to append a new column to each row of an existing csv file. The following code executes what I want:

csv_info = CSV.read("foo.csv")
csv_info.each do |info|
  info << new_object
end

giving me the following in memory when I use pry:

[[uuid0, account0, url0, new_object],
 [uuid1, account1, url1, new_object]]

This doesn't write to file since it's a CSV.read instead of a CSV.open("foo.csv","w"). When I attempt to write over the file with the csv_info object:

csv_info = CSV.read("foo.csv")
csv_info.each do |info|
  info << new_object
end
CSV.open("foo.csv", "w") do |old_csv|
  old_csv << csv_info
end

The file ends up as such:

[["[\"<uuid0>\", \"<account0>\", \"<url0>\", \"<new_object>\"]",
   "[\"<uuid1>\", \"<account1>\", \"<url1>\", \"<new_object>\"]"]]

How can I append without impacting the csv formatting?

`CSV#<<` expects a single row. – Stefan Mar 18 '15 at 15:41 — Stefan, Mar 18 '15 at 15:41

Matt Brictson · Answer 1 · 2015-03-18T18:12:02.307

4

In your example you are reading the entire CSV and manipulating it in memory. Instead, I would suggest reading and modifying one row at a time, especially if the CSV file is large.

Here is how you could do the modification row-by-row to a temp file, then replace the original file with the temp file that was generated.

require "csv"
require "fileutils"
require "tempfile"

temp = Tempfile.new("csv")

CSV.open(temp, "w") do |temp_csv|
  CSV.foreach("foo.csv") do |orig|
    temp_csv << orig + ["new_object"]
  end
end

FileUtils.mv(temp, "foo.csv", :force => true)

edited Mar 18 '15 at 18:12

answered Mar 18 '15 at 16:28

Matt Brictson

10,904
1
38
43

Processing using [line-by-line is as fast as slurping the entire file, and is not prone to scalability issues](http://stackoverflow.com/q/25189262/128421). But, opening the file using `CSV.open` and assigning it to a temporary variable isn't the Ruby way. Use a block so Ruby will automatically close the output file when the block exits. So, +1 for the use of line-by-line IO and -1 for not using a block. Using a temp file is nice, but not necessary, just write to a new file and then `mv` it. – the Tin Man Mar 18 '15 at 17:26
@theTinMan great feedback, thanks. I switched to block style for `CSV.open`. Not sure how I overlooked that! – Matt Brictson Mar 18 '15 at 18:12

score 1 · Accepted Answer · answered Mar 18 '15 at 16:17

CSV#<< expects a single row:

CSV.generate { |csv| csv << [1, 2, 3] }
#=> "1,2,3\n"

CSV.generate { |csv| csv << [1, 2, 3] << [4, 5, 6] }
#=> "1,2,3\n4,5,6\n"

but you're passing a two-dimenional array (which doesn't work as expected):

CSV.generate { |csv| csv << [[1, 2, 3], [4, 5, 6]] }
#=> "\"[1, 2, 3]\",\"[4, 5, 6]\"\n"

So instead of:

old_csv << csv_info

you need something like:

csv_info.each do |info|
  old_csv << info
end

Add new column to each row of a .csv in ruby

2 Answers2