deserializing json with hebrew letters

Question

I am trying to deserializ a json file which contains hebrew words. I am useing .net core.

this is part of my json:

{

    "MandatoryWords": {
        "select": {
            "sql": "select",
            "oracle": "select",
            "Hebrew": [ "בחר", "הצג" ]
        },

        "from": {
            "sql": "from",
            "oracle": "from",
            "Hebrew": [ "מ", "מתוך", "של" ]
        },

        "where": {
            "sql": "where",
            "oracle": "where",
            "Hebrew": [ "כש", "בתנאי ש", "אם", "כאשר", "לכל מי ש", "רק עבור", "עבור" ]
        }
   }
}

I built this class for every part

public class JsonPart
{
    public string sql { get; set; }
    public string oracle { get; set; }
    public List<string> Hebrew { get; set; }
}

When I use this code :

        var options = new JsonSerializerOptions
        {
            PropertyNameCaseInsensitive = true,
        };
        Dictionary < string,Dictionary<string,JsonPart>> j;
        var myJsonString = File.ReadAllText(@"C:\Users\1\Desktop\ConsoleApp1\ConsoleApp1\Substitutes.json");
        j = JsonSerializer.Deserialize<Dictionary<string, Dictionary<string, JsonPart>>>(myJsonString);

Instead of the Hebrew words, the dictionary has question marks like in the following picture

And if I use this code :

        var options = new JsonSerializerOptions
        {
            PropertyNameCaseInsensitive = true,
        };
        Dictionary < string,Dictionary<string,JsonPart>> j;
        var myJsonString = File.ReadAllBytes(@"C:\Users\1\Desktop\ConsoleApp1\ConsoleApp1\Substitutes.json");

        var utf8Reader = new Utf8JsonReader(myJsonString);
        j = JsonSerializer.Deserialize<Dictionary<string, Dictionary<string, JsonPart>>>(ref utf8Reader);

I get this error:

System.Text.Json.JsonException
  HResult=0x80131500
  Message=The JSON value could not be converted to System.Collections.Generic.List`1[System.String]. Path: $.MandatoryWords.select.Hebrew[0] | LineNumber: 6 | BytePositionInLine: 29.
  Source=System.Text.Json
  StackTrace:
   at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& readStack, Utf8JsonReader& reader, Exception ex)
   at System.Text.Json.JsonSerializer.ReadCore(JsonSerializerOptions options, Utf8JsonReader& reader, ReadStack& readStack)
   at System.Text.Json.JsonSerializer.ReadValueCore(JsonSerializerOptions options, Utf8JsonReader& reader, ReadStack& readStack)
   at System.Text.Json.JsonSerializer.ReadValueCore(Utf8JsonReader& reader, Type returnType, JsonSerializerOptions options)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](Utf8JsonReader& reader, JsonSerializerOptions options)
   at Simple.MyConverter.Converter() in C:\Users\1\Desktop\ConsoleApp1\ConsoleApp1\MyConverter .cs:line 31
   at ConsoleApp1.Program.Main(String[] args) in C:\Users\1\Desktop\ConsoleApp1\ConsoleApp1\Program.cs:line 10

  This exception was originally thrown at this call stack:
    [External Code]

Inner Exception 1:
InvalidOperationException: Cannot transcode invalid UTF-8 JSON text to UTF-16 string.

Inner Exception 2:
DecoderFallbackException: Unable to translate bytes [E1] at index 0 from specified code page to Unicode.

I would love a solution.

[not reproducible](https://dotnetfiddle.net/fmnnwM) ... you have some strange encoding in your file (maybe you used `Windows-1255`) — Selvin, Dec 03 '20 at 16:22
In the first link that worked with question marks and diamonds, the code may be working. What is viewed in the Local window is not applying any encoding. — jdweng, Dec 03 '20 at 16:33
I assume this is just a display issue in Visual Studio. Have you tried serializing your model again and saving it to file? If the resulting file has the Hebrew characters it’s definitely just a display issue. — ckuri, Dec 03 '20 at 17:30
Use this method to detect the encoding: https://stackoverflow.com/questions/3710374/get-encoding-of-a-file-in-windows Then change `Utf8JsonReader` to the correct reader for the encoding. — JoelFan, Dec 03 '20 at 18:24

score 1 · Accepted Answer · answered Dec 03 '20 at 17:37

1

open the file Substitutes.json in notepad and save it with UTF-8 encoding, I tested it, after save it with ANSI or UTF-16 its not working

answered Dec 03 '20 at 17:37

NajiMakhoul

1,623
2
16
30

How do I do that? – c sol Dec 03 '20 at 18:24
1

In regular Windows Notepad: Save As > select an encoding at window bottom border > save – Den Dec 03 '20 at 19:33
or install Notepad++ and check Encoding menu option – Den Dec 03 '20 at 19:34

score 1 · Answer 2 · answered Dec 03 '20 at 18:10

1

Check the encoding of your file. Then read it:

var json = File.ReadAllText("path_to_file", "file_encoding");

answered Dec 03 '20 at 18:10

Den

397
3
4

deserializing json with hebrew letters

2 Answers2