Saving CR/LF in json data

Question

I am saving table data to a json object. The table data is coming from txt inputs and textareas in the table cells.

I'm running into a problem with CR/LF characters in the JSON elements holding the textarea data. The JSON data gets saved to the database fine, but when I pass it back to the jQuery function that populates the table using that data, I get this:

SyntaxError: JSON.parse: bad control character in string literal at line 1 column 67 of the JSON data
var array = JSON.parse(notes),

in the console.

I put the JSON data in Notepad++ with Show All Characters on and the CR/LF was at column 67.

Here's a sample of JSON data that I'm working with:

[["","","",""],["","9/23/14","",""],["","30789 detail, x_vendor_no**CR/LF HERE**
20597 header","",""],["","99 del invalid x_vendor_no","",""],["","30780","",""],["","","",""],["","","",""],["","","",""]]

Is there a way to allow CR/LF in the data?

UPDATE 11684's suggestion to use replace to remove the \r part of the CRLF won't work. Here's why:

Here's the complete function that uses the JSON data: (Updated to work with Update #2 code below)

function PopulateTaskTableWithNotes(tableID,notesArray) {
    // JSON parse removed per answer suggestion
    var r, c, note;

    for (r = 0; r < notesArray.length; ++r) {
        for (c = 0; c < notesArray[r].length; ++c) {
            note = notesArray[r][c];
        $('#' + tableID + ' tr:nth-child(' + (r + 1) + ') td:nth-child(' + (c + 1) + ')').children(':first-child').val(note);
        }
    }
}

I still get the error on the line that tries to parse the JSON data. The replace function apparently can't "find" characters within an array element.

UPDATE #2 Here's how I am creating the array:

var siteID = $('#ddlUserSites option:selected').val(),
    numRows = $('#' + tableID + ' tr').length,
    numCols = $('#' + tableID).find('tr:first th').length,
    notesArray = new Array(numRows),
    rowNum = 1,
    note = '',
    colNum;

while (rowNum <= numRows) {
    notesArray[rowNum] = new Array(numCols);

    // Reset colNum for next row iteration
    colNum = 1;
    while (colNum <= numCols) {
        note = '';

        if ($('#' + tableID + ' tr:nth-child(' + rowNum + ') td:nth-child(' + colNum + ')').children(':first-child').is('input,textarea')) {
            note = $('#' + tableID + ' tr:nth-child(' + rowNum + ') td:nth-child(' + colNum + ')').children(':first-child').val();
        }
        notesArray[rowNum][colNum] = note;
        //console.log('Note for rowNum ' + rowNum + ', colNum ' + colNum + ': ' + note);
        colNum++;
    }
    // Remove first element in current row array
    notesArray[rowNum].shift();
    rowNum++;
}
// Remove first element in array
notesArray.shift();
JSON.stringify(notesArray); // Added per an answer here
console.log('Final notesArray: ' + $.toJSON(notesArray));

$.ajax({
    data: {saveTaskNotes: 'true', userID:userID, siteID:siteID, taskTable:tableID, notes:notesArray},
    success: function(data) {
        console.log('Save task notes data: ' + data);
    }
});

The "Final notesArray" console output looks fine, but now, with stringify added, the function above (PopulateTaskTableWithNotes) console output shows that it's reading through every character in the array as a separate element!

Maybe this will help too, as far as what's happening to the data between the creating and reading functions: the array is being saved to a single MySQL database field and then retrieved for the PopulateTable function via $.ajax() (on both ends).

Having said that, do I need to look at what I'm doing with/to the array in the PHP code?

UPDATE #3 Here's the PHP function that takes the data in and writes to the MySQL db:

function SaveTaskNotes($userID,$siteID,$taskTable,$notes) {
    $notes = json_encode($notes);
    $insertUpdateTaskNotesResult = '';
    $insertTaskNotes = "INSERT INTO userProgress (userProgressUserID,userProgressSiteID,userProgressNotesTable,userProgressNotes) values ($userID,$siteID,'" . $taskTable . "','" . $notes . "')";
    $log->lwrite('$insertTaskNotes: ' . $insertTaskNotes);
    $resultInsertTaskNotes = @mysqli_query($dbc,$insertTaskNotes);
    if ($resultInsertTaskNotes) {
        $insertUpdateTaskNotesResult = 'insertTaskNotesSuccess';
    } else {
        if (mysqli_error($dbc) != '') {
            $log->lwrite('INSERT TASK NOTES: An error occurred while attempting to add the task notes. Query: ' . $insertTaskNotes . ', mysqli_error: ' . mysqli_error($dbc));
        }
        $insertUpdateTaskNotesResult = 'insertTaskNotesFail';
    }
    echo $insertUpdateTaskNotesResult;
}

And here's the function that gets the data from the db and sends it to the above $.ajax function:

function GetUserTaskNotes($userID,$siteID,$taskTableID) {
    $queryGetUserTaskNotes = "SELECT userProgressNotes FROM userProgress WHERE userProgressUserID = $userID AND userProgressSiteID = $siteID AND userProgressNotesTable = '" . $taskTableID . "'";
    $log->lwrite('$queryGetUserTaskNotes: ' . $queryGetUserTaskNotes);
    $resultGetUserTaskNotes = @mysqli_query($dbc,$queryGetUserTaskNotes);
    if ($resultGetUserTaskNotes) {
        $taskNotes = mysqli_fetch_assoc($resultGetUserTaskNotes);
        $log->lwrite('Retrieved $taskNotes[\'userProgressNotes\']: ' . $taskNotes['userProgressNotes']);
        echo $taskNotes['userProgressNotes'];
    } else {
        if (mysqli_error($dbc) != '') {
            $log->lwrite('GET TASK NOTES: An error occurred while attempting to retrieve the task notes. Query: ' . $queryGetUserTaskNotes . ', mysqli_error: ' . mysqli_error($dbc));
        }
        echo 'getTaskNotesFail';
    }
}

In both the save and get functions the $log output shows that the array never changes (with the above js/php code) and pasting the array in to notepad++ shows that the CR/LF is still there throughout.

Quick fix: `notes.replace("\r\n", "");`. I wouldn't recommend that though. — 11684, Sep 24 '14 at 13:54
@11684: I'm hoping to find a solution that will allow me to retain those new lines, or in some way repopulate the textarea with the newlines where the user entered them. — marky, Sep 24 '14 at 13:57
Yes, when I wrote the comments I hadn't seen your edit including the example data yet, and I (mistakenly) assumed the newlines were not part of the data, but somewhere between the values (so, safe to remove). Alternative: `notes.replace("\r\n", "\n");`. Windows is the only (mainstream) OS using `\r\n` instead of `\n` for linebreaks, so a browser wouldn't (shouldn't) mind. — 11684, Sep 24 '14 at 14:01
I don't know if this is causing the error, but you wrote `.replace("\r\n", "n");` instead of `.replace("\r\n", "\n");`. — 11684, Sep 24 '14 at 14:37
I fixed the replace, that didn't fix the problem :( I'm certain it's because it has something to do with the fact that I'm using a replace on an array, not a string. — marky, Sep 24 '14 at 14:42
you are doing things unnecessary twice!!! data on $.ajax do not need to be JSON.sringify(ed), jQuery cares that. but if you do, do not json_encode it again on PHP side when you recive that data. — Saic Siquot, Sep 24 '14 at 19:36
Makes sense and my original code didn't have that stringify line - I added that from a suggestion here... Anyway, I commented that out but now when the array is iterated through to populate the table, each character is treated as a separate array element. — marky, Sep 24 '14 at 19:44

Daniel W. · Accepted Answer · 2014-09-24T20:07:46.643

3

Don't use JSON.parse, the data is already JSON and Javascript can work with it.

You only need it when passing a string, imagine JSON.parse() beeing like string2json().

I think this might already be a solution to your problem, I've never had issues with new line characters.

As Luis said, the problem is not your client (Javascript, jQuery), besides the JSON.parse, but the providing site is wrong.

Example for PHP:

<?php
echo json_encode(array("test" => "


x"));

PHP properly escapes the characters:

{"test":"\r\n\r\n\r\nx"}

But the source of your data is providing malformed JSON.

To fix the JSON issue, either use prepared statements or use:

$notes = str_replace('\', '\\', json_encode($notes)); // in SaveTaskNotes

edited Sep 24 '14 at 20:07

answered Sep 24 '14 at 15:05

Daniel W.

31,164
13
93
151

1

You said: "Don't use JSON.parse, the data is already JSON and Javascript can work with it." While this makes sense and the console shows that to be the case when I look at it when it's coming to the function from the php script, if I DON'T use parse, the array is being interpreted as each character is its own array element. Also, using JSON.parse(notesArray) when there are no new line characters in it works just fine! So what's the solution? – marky Sep 24 '14 at 18:45
Please add the PHP code which comes after the ajax call (posting the json) and the PHP code before trying to ajax the JSON. Something is wrong short before or after the database interaction. – Daniel W. Sep 24 '14 at 18:50
@eventide in the database, through phpMyAdmin, in the column `userProgressNotes `, do you see `\r\n` literally or do you see real line breaks? `userProgressNotes ` contains JSON right? – Daniel W. Sep 24 '14 at 19:37
phpMyAdmin displays the contents of the field with the actual line breaks - not "\r\n". userProgressNotes is a varchar(5000) field. – marky Sep 24 '14 at 19:50
@eventide you should use prepared statements, which would save you a lot of trouble here. Try this as a workarround: `$notes = str_replace('\', '\\', json_encode($notes));` in `SaveTaskNotes` - let me know if it works. – Daniel W. Sep 24 '14 at 19:52
That prevented the error AND got the table populated. So that's fixed - Thanks! I noticed, however that the console output for notesArray.length is still the count for each of the characters in the array elements, even though the values got to the correct table cells. Any idea about this behavior? – marky Sep 24 '14 at 20:02
@eventide that means that `notesArray` is not an array in that moment. This is where the magic mentioned earlier comes in: `notesArray = JSON.parse(notesArray);` - after that it should be an array and not a string. – Daniel W. Sep 24 '14 at 20:08
Sounds good - thanks for your time! I'll take a good look at prepared statements, too! :) – marky Sep 24 '14 at 20:12
1

No problem! I'm not sure about the prepared statements would prevent the new line characters from beeing escaped... but they are really better. Good luck and have fun coding :) – Daniel W. Sep 24 '14 at 20:14

score 1 · Answer 2 · answered Sep 24 '14 at 14:43

1

Well, the error is on the input data (showed in question). You can't have an CR or LF inside a literal in a JSON string. What you can have are that chars escaped as \r \n. The problem is on other side, where escaped codes are replaced by actual chars and therefore the full JSON string becomes invalid.

answered Sep 24 '14 at 14:43

Saic Siquot

6,513
5
34
56

Makes sense. How would I escape the CR and LF when that element of the array is being added? – marky Sep 24 '14 at 14:48
apply `JSON.stringify(values);` to the whole data, not by parts over elements. you will get an unique string that codifies all. you will get cr & lf properly codified also. there are tons of examples. [-1-](http://stackoverflow.com/questions/6810084/encoding-javascript-object-to-json-string) [-2-](http://stackoverflow.com/questions/10919965/how-do-i-encode-a-javascript-object-as-json) – Saic Siquot Sep 24 '14 at 15:02
I used JSON.stringify on the array after it's done iterating through the table data when the array is created. Console output looked fine (pretty much exactly like it did before adding stringify). And per your suggestion I removed json.parse from where it reads through the array. I didn't get an error that time, but the console output on each time it iterated through the array elements showed that it was separating out each and every single character as an array element! I've added my array building script to my question. – marky Sep 24 '14 at 15:44

Saving CR/LF in json data

2 Answers2