First property inaccsesible of objects parsed by csv-parse

Question

I am parsing a csv file with following contents using csv-parse -

userID,sysID
20,50
30,71

However, on the objects returned it isn't possible to access the property created from the first column userID.

Here is my code --


async function main(){
    let systemIDs = await getSystemIds('./systems.csv');     
    console.log(`Scanning data for ${systemIDs.length} systems..`);

    console.log(systemIDs[0]);
    console.log(systemIDs[0].userID); // This prints undefined 
    console.log(systemIDs[0].sysID);  // This prints the correct value
}

async function getSystemIds(path){
    let ids= [];
    await new Promise ((resolve,reject)=>{        
        const csvParser = csvParse({columns:true, skip_empty_lines: true});
        FS.createReadStream(path)
        .pipe(csvParser)
        .on('readable', ()=>{
            let record ;            
            while(record = csvParser.read()) {
                ids.push(record);
            }            
        })
        .on('finish',()=>{
            resolve();
        });
    });
    return ids;
}

Output -

Scanning data for 2 systems..
{ 'userID': '20', sysID: '50' } 
undefined  // <== The Problem
50

I notice the first column key userID has single quotes around it in the console output where as sysID doesn't. But don't know what is causing them.

Also, can you console.log(record) after: while(record = csvParser.read()) { — Skarlinski, Feb 27 '21 at 20:33
@Skarlinski it looks the same `{ 'userID': '20', sysID: '50' }` — charsi, Feb 27 '21 at 21:06
sounds like some weird unicode or hidden spaces issue in the original csv — Skarlinski, Feb 27 '21 at 22:14
I would test other csv's, or add a breakpoint while running with inspect. — Skarlinski, Feb 27 '21 at 22:15
You were on the right track BOM characters generated by excel were to blame — charsi, Feb 27 '21 at 23:47

charsi · Accepted Answer · 2021-02-28T00:17:19.223

Figured it out myself in the end...

I needed the BOM option. The documentation states it should be set to true for UTF-8 files. But it defaults to false.

Excel by default generates csv files with BOM as the first character in CSV files. This gets picked up as part of the header (and key name) by the parser. With the bom option set to true, it can handle csv files generated from excel or other programs.

const csvParser = csvParse({
  columns: true, 
  skip_empty_lines: true,
  bom: true
});

First property inaccsesible of objects parsed by csv-parse

1 Answers1

Linked