3

I know there are other posts out there but none of them seem to fix my issues. I am using csv-parse with node js. This is the CSV header and record that I'm trying to parse.

sku,description,productUnitOfMeasure,unitCost,retailPrice,wholesalePrice,dropShipPrice,assemblyCost,planner,comments,productId,fileUpdate,SkuStatus,Master Planning Family,Category,Sub-Category,Brand,ShortCode,Import/Domestic,Inventory Value,Master Pack Quantity,Pallet / TI / HI,40HC Quantity,Product Group ID
032406021945-GreenBay,TFAL B2080264 INIT TNS GRY SAUTE PN 8",EA,7.72,13.99,0.00,0.00,0,Whitney Ehlke-2307,,032406021945,2022-01-25,New,COOKWARE,OPENSTOCK,NONE,T-FAL,B2080264,Domestic,208.44,3,0/0/0,0,23

I have no control over this file. I just need to be able to parse it. You will see that there is a double quote at the end of the description: TFal B2080264 INI TNS GRY SAUTE PN 8".

I need the double quote to stay there and for that to parse as one field. I keep getting this error:

Invalid Opening Quote: a quote is found inside a field at line 2.

The quote is not an opening. It's technically a closing. But regardless, it will not parse.

This is currently my code:

const parser = fs.createReadStream(filePath).pipe(
    parse({ columns: true, relax_quotes: true, escape: '\\', ltrim: true, rtrim: true })
)

I have removed some of the params and tried others, to no avail. Any ideas??

trincot
  • 317,000
  • 35
  • 244
  • 286
Just Me
  • 91
  • 4
  • 12
  • In case it's not clear to readers, the `8"` is part of the product description and indicates that the sauté pan is 8 inches in diameter. – jarmod Jan 27 '22 at 14:35

2 Answers2

9

This code works fine with the latest csv-parse version (5.0.4). Which version of the csv-parse package are you using? I ask because it looks like the option may have been renamed from relax to relax_quotes only recently.

So, I think the solution is either:

  1. upgrade to the latest csv-parse, and indicate relax_quotes, or
  2. stay with your current version of csv-parse, and indicate relax

Just to be sure relax_quotes works with the current library, I tested the following code and it worked as expected:

const csv = require('csv-parse');
const fs = require('fs');

const parser = fs.createReadStream("70880341.csv").pipe(
  csv.parse({ columns: true, relax_quotes: true, escape: '\\', ltrim: true, rtrim: true })
)

const records = [];

parser.on('readable', function() {
  let record;
  while ((record = parser.read()) !== null) {
    records.push(record);
  }
});

parser.on('error', function(err) {
  console.error(err.message);
});

parser.on('end', function() {
  console.log(records);
});

Result:

[{
    sku: '032406021945-GreenBay',
    description: 'TFAL B2080264 INIT TNS GRY SAUTE PN 8"',
    productUnitOfMeasure: 'EA',
    ...
}]
jarmod
  • 71,565
  • 16
  • 115
  • 122
  • 1
    I had already tried the code you have, but I was using the new relax-quotes param. Changing it to just relax worked! Thanks so much! – Just Me Jan 27 '22 at 22:14
  • Note that ltrim and rtrim did error with some records. Removing them has worked for all records so far. – Just Me Jan 30 '22 at 00:49
1

For me full error message was: CsvError: Invalid Opening Quote: a quote is found on field 0 at line 1, value is "" (utf8 bom)

And adding "bom: true" parameter for csv.parse - helped.

PokatilovArt
  • 1,111
  • 11
  • 13