Can't use JSON.parse() to import array from PHP

Question

So I've got a form where I can create multiple datalist-text-inputs of the same type that is later (onclick) put into an invisible input before submitted.

These datalist-text-inputs is created onload so that I can add more with an "+" button with the same function.

The options are imported from my PHP-database, and it worked fine. But suddenly it stopped working and I don't know why. I've tried a thousand things but can't figure it out.

I'm quite new to PHP. I think the problem has to do with JSON.parse() since the code breaks on that line.

script.js

var ajax = new XMLHttpRequest();
ajax.open("GET", "fetch data.php", true);
ajax.send();
ajax.onreadystatechange = function() {
    if(this.readyState == 4 && this.status == 200) {
        var data = JSON.parse(this.responseText);
        var html = "";
        for (var a = 0; a < data.length; a++) {
            var firstName = data[a].name;
            html += "<option value='" + firstName + "'></option>";  
        };
        document.getElementById(type+"list"+addnumber).innerHTML = html;
    };
};

type+"list"+addnumber it the name of the input-text-box. Type is an argument and addnumber an variable/integer.

fetch data.php

<?php
    $host = "localhost"; $user = "root"; $pass = ""; $db = "collection"; 
    $conn = mysqli_connect($host, $user, $pass, $db);
    $result = mysqli_query($conn, 'SELECT name FROM musicians ORDER BY name');
    $data = array();

    while ($row = mysqli_fetch_assoc($result)) {
        $data[]=$row;
    };
    
    echo json_encode($data);
?>

Also, I might add that this function creates objects in three places on the same page but the value is added/moved to three different invisible inputs.

Answer 1

Based on the exception you're seeing, ie:

JsonException: Malformed UTF-8 characters, possibly incorrectly encoded in C:\wamp64\www\collection\fetch data.php on line <i>11</i>

My guess is that the data you are reading is not encoded in a way which json_encode expects.

The simplest (NOT RECOMMENDED) approach is to pass in the JSON_INVALID_UTF8_IGNORE or JSON_INVALID_UTF8_SUBSTITUTE flags, which will cause bad data to be silently skipped over (or replaced with the unicode REPLACEMENT CHARACTER \0xfffd), rather than treating it as an error. See the documentation for json_encode and predefined JSON constants.

If you want to get to the root of the problem so that all data is correctly encoded as JSON, however:

You can try to force the encoding by setting the mysql character set using PHP's mysqli_set_charset function, eg:

<?php
    $host = "localhost"; $user = "root"; $pass = ""; $db = "collection"; 
    $conn = mysqli_connect($host, $user, $pass, $db);
    if (!mysqli_set_charset($conn, 'utf8')) {
        throw new \Exception("failed to set mysql charset");
    }
    $result = mysqli_query($conn, 'SELECT name FROM musicians ORDER BY name');
    ...
?>

The most common charsets in my experience are utf8, utf8mb4. If you know your data to contain some other specific character set, you may need to translate it into utf8 before trying to encode it into JSON using PHP's mb_convert_encoding function.

Finally, it could be that the issue has occurred earlier in your application, resulting in bad (mixed-encoding) data. If this is the case, you'll need to detect the bad data row-by-row, perhaps outputting where exceptions were raised in a separate error report, and manually correct the encoding of that data. This can be prevented by ensuring the data is validated and correctly encoded as utf8 before it reaches your database. Note that ensuring the mysql character set is correctly set for all connections is also potentially a part of this solution. It may be that you'll want to configure your database to do this automatically.

example of detection and logging:

<?php
    $host = "localhost"; $user = "root"; $pass = ""; $db = "collection"; 
    $conn = mysqli_connect($host, $user, $pass, $db);
    if (!mysqli_set_charset($conn, 'utf8')) {
        throw new \Exception("failed to set mysql charset");
    }
    $result = mysqli_query($conn, 'SELECT name FROM musicians ORDER BY name');
    $data = array();

    while ($row = mysqli_fetch_assoc($result)) {
        try {
            // detect rows which cannot be encoded
            $discard = json_encode($row, JSON_THROW_ON_ERROR);
        } catch (\JsonException $e) {
            // keep track of failed rows so we can correct them later
            my_custom_logging_function($row);

            // and skip the row so we don't try to encode it later
            continue;
        }
        $data[]=$row;
    };

    echo json_encode($data, JSON_THROW_ON_ERROR);
?>

Answer 2

OK just change "fetch data.php" to "data.php"