Extra newline characters in JavaScript arrays

Question

I am working on a program that parses a CSV files and then moves the respective data into different arrays that will make a spreadsheet out of them. However, my spreadsheet has extra blank rows with one cell each. Upon further inspection, I have found these extra cells contain a single newline character each. I have filtered my arrays so that they do not contain any newline characters, yet the problem still persists. I am doing this in conjunction with AngularJS, and while I doubt that is causing the problem, I did not have this issue before I implemented Angular.

Has anyone had anything like this before? With dynamic creation of newline characters where they shouldn't be.

Here is some of my code:

        headers = breakText[0];
        function isDefined(element, index, array){
            return (element !== undefined || element !=='\n');
            //I tried to stop the newlines here, but this did nothing
        };


        //new header file including ONLY desired titles; no undefines
        $scope.clearHeaders = headers.filter(isDefined);
        $scope.clearPatients = [];
        for(var i=1; i<breakText.length; i++){
            if breakText[0] {
                $scope.clearPatients.push(breakText[i].filter(isDefined));
            }
        };

breakText was a 2D array containing undefined areas where I had deleted stuff. So the above code is creating a new header array and another, 2D array to hold an array of data for each person, all without undefined spaces. These arrays contain blanks "" as well as data, but nothing with \n or \r\n.

Here is part of the AngularJS implementation:

$scope.columns = $scope.clearHeaders;
$scope.cells = {};
$scope.values = $scope.clearPatients.map(function(c,row){
    return c.map(function(data){
        return {
            content: data,
            color: $scope.makeColors(data)
        };
    });
});

We have tried using a replace function to replace \n with '' and disallow it to return the data if it is '' but this causes issues since some of our initial data is also blank. It also does not stop the creation of the new cell. We have also tried wrapping it in something such as if(data !== '\n') but that changed nothing.

Edit1:

You can see it working live here, yet it is designed to work with this small CSV file that can be downloaded here. The text below the table after you upload a file shows what each cell is bound to, and if you click and edit the cells, you can see the content dynamically changing. By clicking on the extra cells between the rows, you can see that they contain a newline character.

I just tried creating a new filter function, where it would check for a newline in the clearPatients array, and then filter everything into a new array, sans-newline. This new array was then called to be in the table rather than the one with newlines. Yet this threw many errors that could not be determined of their source.

Edit2:

After Mike P's answer and some other input, it was determined that the best way to fix this problem would be to swap the line var allTextLines = $scope.csv.split(/\r\n|\n/); with the regex /\r?\n/

Thanks for your help!

Any chance you can provide a JSFiddle or Plnkr that demonstrates what's happening? — Mike Pugh, Jul 18 '13 at 17:51
You've got some new lines in your allTextLines variable on line 31 of spreadsheetController.js after it splits the CSV. You'll want to check your regex or at least loop through that string array and wipe out those non-desired lines. I'll try to post a screenshot of the debug. — Mike Pugh, Jul 18 '13 at 18:35
I'd urge you to take a look at that link I posted about the JS code to parse CSV data. Many have discussed why the split() method shouldn't be used for this task, and there are several implementations of CSV parsers provided that could make your life a lot easier. — Mike Pugh, Jul 18 '13 at 19:13

score 1 · Accepted Answer · edited May 23 '17 at 12:29

Your problem doesn't look like it has anything to do with AngularjS. You've got your text data in $scope.csv and you split off your regular expression, and that creates an array of strings in allTextLines.

This array of strings has some useless lines in it (at idx 1, 3, and 4). So you'll either want to modify your regex split expression or loop through this array and remove stuff you don't want.

You may want to check out this guy's CSV to Array JS code @ Javascript code to parse CSV data

I can clearly see your spreadsheet code creating rows & cells for these portions of allTextLines. So remove those and you're in business.

debug of code

For what it's worth, it looks like using split() is one of the most despised ways of parsing CSV data. — Mike Pugh, Jul 18 '13 at 18:44

Extra newline characters in JavaScript arrays

1 Answers1