How to get a UTF-8 JSON

Question

I'm using LitJSON library but things gets a little bit odd.

Do you know any JSON library that keeps the accents when converting ?

Here's the test :

test.json

[{"id":"CS_001","name":"L'élément","type":"Tôt"},{"id":"CS_002","name":"L'outrage","type":"Tôt"},{"id":"CS_003","name":"Test","type":"Tôt"}]

test.cs

public class test : MonoBehaviour {
    private string jsonString;
    private JsonData cardData;
    JsonData database;

    void Start () {
        jsonString = File.ReadAllText (Application.dataPath + "/test.json");
        cardData = JsonMapper.ToObject (jsonString);
        database = JsonMapper.ToJson (cardData);
        Debug.Log (database.ToString ());
    }
}

And the Debug.Log turns to :

[{"id":"CS_001","name":"L'\u00E9l\u00E9ment","type":"T\u00F4t"},{"id":"CS_002","name":"L'outrage","type":"T\u00F4t"},{"id":"CS_003","name":"Test","type":"T\u00F4t"}]

Any idea how to get a proper Json ? Even if it's with another JSON library.

Thank you very much.

Try using [Json.net](http://www.newtonsoft.com/json). – Nasreddine Oct 13 '15 at 12:35 — Nasreddine, Oct 13 '15 at 12:35

score 4 · Answer 1 · answered Oct 13 '15 at 12:36

4

Content-type: application/json; charset=utf-8 designates the content to be in JSON format, encoded in the UTF-8 character encoding. The default encoding for JSON is UTF-8. In this case the receiving server apparently does not know that it's dealing with JSON in the UTF-8 encoding and you may need to convert it manually:

byte[] encodedBytes = Encoding.UTF8.GetBytes(jsonString);
Encoding.Convert(Encoding.UTF8, Encoding.Unicode, encodedBytes);

or just try to specify the content type on your request:

content-type: application/json; charset=utf-8

answered Oct 13 '15 at 12:36

dlght

1,406
1
18
35

Hi, thank you for you answer, I tried the byte[] but that's not working. I'm on mono/unity and didn't found where to specify the content type, thank you – MedianP Oct 18 '15 at 18:52
1

Just a comment here, if you're getting a json string from a stream and using a stream reader to "ReadToEnd" then .net will default to read as Unicode instead of the UTF8 the data is in, which will very often cause a catastrophic and silent bug that may be quite hard to figure out. To work around, one can either set the Encoding parameter on the `StreamReader` constructor or read the stream into a `byte[]` and decode with `Encoding.UTF8.GetString` – Felype Apr 03 '18 at 17:19

score 3 · Accepted Answer · answered Oct 13 '15 at 13:17

Here's an example using Json.Net to deserialize the string:

using System;
using Newtonsoft.Json;
using System.Collections.Generic;

public class Program
{
    public static void Main()
    {
        // Deserialize the JSON into a list of CardData
        var ob = JsonConvert.DeserializeObject<List<CardData>>("[{\"id\":\"CS_001\",\"name\":\"L'élément\",\"type\":\"Tôt\"},{\"id\":\"CS_002\",\"name\":\"L'outrage\",\"type\":\"Tôt\"},{\"id\":\"CS_003\",\"name\":\"Test\",\"type\":\"Tôt\"}]" );

        /*
          The output will be:
            id: CS_001, name: L'élément, type: Tôt
            id: CS_002, name: L'outrage, type: Tôt
            id: CS_003, name: Test, type: Tôt
        */
        foreach(var i in ob){
            Console.WriteLine(i);  
        }
    }
}

// Class that will hold the deserialized data
// For demo puposes
public class CardData
{
    public string id { get; set; }
    public string name { get; set; }
    public string type { get; set; }

    public override string ToString(){
        return String.Format("id: {0}, name: {1}, type: {2}",id, name, type);   
    }
}

Live demo available here

score 0 · Answer 3 · answered Oct 13 '15 at 12:33

0

It is possible that you read the text file using the wrong encoding. Try to use the overload for File.ReadAllText that takes in an Encoding argument and pass it UTF8.

answered Oct 13 '15 at 12:33

Rune Grimstad

35,612
10
61
76

Hi, thank you for your answer, I tried : File.WriteAllText (Application.dataPath + "/database.json", database.ToString(), Encoding.UTF8); but it stays the same. – MedianP Oct 18 '15 at 18:53

How to get a UTF-8 JSON

3 Answers3

Linked