0

I am trying to read CSV file in JavaScript using npm module csv-to-array into an array. I expect the rows_in_csv array to hold all rows (count > 1000) of CSV. But it holds data for only first 507 rows. This is the issue.

let fs = require('fs');
let cwd = __dirname;
let file_path = cwd.concat("/farsi_localization_data.csv");

require("csv-to-array")({
  file: file_path,
  columns: ["Complete key", "", "English Value", "Hindi Value"]
}, function (err, rows_in_csv) {
  let row_count = rows_in_csv.length;

  console.log(`number of rows in CSV is ${row_count}`);
});

Please tell why this would be happening.

Nishant Kumar
  • 363
  • 4
  • 20
  • 1
    please give a sample file u are trying – PRAJIN PRAKASH Mar 18 '20 at 13:00
  • 1
    Is there anything notably different about row #507 in the `.csv` file? – RobC Mar 18 '20 at 13:07
  • @PRAJINPRAKASH stackoverflow does not allow file upload. please check this: https://docs.google.com/spreadsheets/d/1VjdYuTuCN6VEdQoQZQl9hHP0oJSijMeQFn9isxXZFHg/edit?usp=sharing – Nishant Kumar Mar 18 '20 at 13:25
  • Row no. 507 seems to be the first row that has a cell with line break in it. – RobC Mar 18 '20 at 13:38
  • 1
    @NishantKumar yes please remove newlines from the file on line 507 and 508. then you may get an error `FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory` just do [it](https://stackoverflow.com/questions/26094420/fatal-error-call-and-retry-last-allocation-failed-process-out-of-memory) – PRAJIN PRAKASH Mar 18 '20 at 14:09
  • yes. i removed newlines. after that got the above error! now trying to fix that.. @PRAJINPRAKASH – Nishant Kumar Mar 19 '20 at 09:54
  • @NishantKumar Best way is do split csv into small files. Try node --max-old-space-size=4076 filename.js – PRAJIN PRAKASH Mar 19 '20 at 12:13
  • hmm "node --max-old-space-size=4076 update-farsi-json-file.js" did not work. finally i wrote ruby script to do this job :) – Nishant Kumar Mar 19 '20 at 12:55
  • splitting the CSV file into small files may also solve the issue. but the fact that we are unable to read a CSV with 2000 rows only using above code is bad! – Nishant Kumar Mar 19 '20 at 17:23

0 Answers0