Stack Overflow
Stack Overflow

Questions tagged [smartercsv]

29 questions
5
votes
1 answer

How do I use Ruby to combine several CSV files into one big CSV file?

I have been using SmarterCSV to convert bed format file to csv file and changing the column names. Now I have collected several CSV files, and want to combine them into one big CSV file. In test3.csv, there are three columns, chromosome,…
Michael
  • 65
  • 1
  • 7
4
votes
2 answers

Controller not recognizing installed Gem

Okay, I know this is a bit of a simple question, but I can't seem to get it to work. I have installed the SmarterCSV gem in my Rails 4 app and am trying to use it in my controller like so: SmarterCSV.process("/files/csv_file.csv") I can do this…
Zubatman
  • 1,235
  • 3
  • 17
  • 34
3
votes
3 answers

Ruby: If column exists use it, if not add to other column

I'm parsing a CSV and trying to distinguish between columns in Model and "virtual" columns that'll be added to a JSONB :data column. So far I've got this: rows = SmarterCSV.process(csv.path) rows.each do |row| row.select! { |x|…
t56k
  • 6,769
  • 9
  • 52
  • 115
3
votes
2 answers

SmarterCSV and file encoding issues in Ruby

I'm working with a file that appears to have UTF-16LE encoding. If I run File.read(file, :encoding => 'utf-16le') the first line of the file is: "=\"25/09/2013\"\t18:39:17\t=\"Unknown\"\t=\"+15168608203\"\t\"Message.\"\r\n If I read the…
AvocadoRivalry
  • 411
  • 1
  • 7
  • 16
3
votes
2 answers

Cannot parse a CSV file - "CSV::MalformedCSVError: Unquoted fields do not allow \r or \n"

I am trying to parse uploaded CSV files. Here's the full error message: [Worker(host:PC.local pid:69594)] Job ImportJob (id=4) FAILED (3 prior attempts) with CSV::MalformedCSVError: Unquoted fields do not allow \r or \n (line 1). And the CSV files…
user984621
  • 46,344
  • 73
  • 224
  • 412
2
votes
1 answer

Is there a way to ignore columns with empty headers when using SmarterCSV?

I have a client who wants to maintain a broad span of flexibility when submitting CSV files to be uploaded. I've been using SmarterCSV to process the files, but have hit a snag where there are gaps in the columns of information--essentially the…
Blaine M. Lawson
  • 106
  • 1
  • 5
2
votes
1 answer

CSV::MalformedCSVError: Illegal quoting in line 1 with SmarterCSV

I have an issue, when trying to process a csv file, using SmarterCSV. The error I get is - CSV::MalformedCSVError: Illegal quoting in line 1 This is where the code I use to process the csv file SmarterCSV.process(file_path) I have gone through…
Rajasree Muraleedharan
  • 161
  • 1
  • 9
2
votes
1 answer

Quoted fields escaped in smarter_csv

I have a CSV file that I'm trying to process with smarter_csv in Ruby. Every field has double quuotes and there are some fields that have double quotes nested within them that are not escaped. I'm using :force_simple_split => true as suggested in…
Adam
  • 1,755
  • 20
  • 31
1
vote
0 answers

SmarterCSV end of file reached, but ruby's CSV library can process the file

I have two CSV files which are basically the same, but for some reason, SmarterCSV can not read the one named bad_file Here is a Gist of both files. Ruby's native CSV library can read bad_file no problem. Before processing each file. I strip…
Arel
  • 3,888
  • 6
  • 37
  • 91
1
vote
1 answer

Smarter CSV ignore blank lines in csv

I am using Smarter CSV to and have encountered a csv that has blank lines. Is there anyway to ignore these? Smarter CSV is taking the blank line as a header and not processing the file correctly. Is there any way I can bastardize the comment_regexp?…
i cant code
  • 305
  • 2
  • 7
1
vote
1 answer

Reference column number using smarter_csv in Rails 5

I'm currently using the smarter_csv gem to import data from CSVs to my Rails application. Ideally, I'd like to access the columns imported by number instead of the header name, as the header names could be somewhat inconsistent on the data coming…
Ron M
  • 773
  • 2
  • 10
  • 22
1
vote
1 answer

Csv import error through smartercsv gem

I have this csv structure "CATEGORY";"NAME";"AGE" "Red";"John";"34" When I import the file through smarter_csv gem I get this hash {:"\"CATEGORY\""=>"\"Red\"", :"\"NAME\""=>"\"John\"", :"\"AGE\""=>"\"34\""} The code I use is the following options…
Jack
  • 497
  • 5
  • 16
1
vote
1 answer

use smarter csv gem and processing csv in chunks - i need to delete rows from a large csv ( 2GB) by comparing the key/values with another csv (1 GB)

The following is the code i have used. I am not able to delete the rows from Main.csv, when the value of "name" col in Main.csv equals to the value of "name" col in Sub.csv. Please help me on the same. I know i am missing something. Thanks in…
Hayz
  • 53
  • 12
1
vote
1 answer

Ruby smarter_csv header issues

I am trying to create a Ruby file that extracts data. I seem to run into an issue as the provided CSV files don't have headers attached. So, I am trying to tell smarter_csv that there are NO headers. The code, I use for this is: transactions =…
user2164689
  • 395
  • 4
  • 13
1
vote
1 answer

cannot load such file -- smarter_csv (LoadError)

I want to use smarter_csv gem, but when I write in my code require "smarter_csv" it gives me error message below: /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require':…
arthur-net
  • 1,138
  • 1
  • 13
  • 34
1
2