Encode UTF8 text to Unicode C#

Question

How to encode UTF8 text to Unicode?

string text_txt = "Ð¿ÑÐ¸Ð²ÐµÑ";    
byte[] bytesUtf8 = Encoding.Default.GetBytes(text_txt);
text_txt = Encoding.UTF8.GetString(bytesUtf8);

The problem is output: п�?иве�

I need output: привет

Using that site: https://www.branah.com/unicode-converter enter text in "UTF-8 text (Example: a ä¸ Ð¯)" to "Ð¿ÑÐ¸Ð²ÐµÑ" it will show you "привет" on Unicode text

Please give some advice thanks

No difference by the way. if use it output: 뿐㿑룐닐뗐� — J.Col, Feb 20 '20 at 08:20
I don't think that text is in UTF8. Try using [this site](https://2cyr.com/decode/?lang=en) to find out what encoding that text is in. — Sweeper, Feb 20 '20 at 08:26
original `text_txt` is *not* UTF8, but native C# encoding. see this answer https://stackoverflow.com/a/11300580/21336 — devio, Feb 20 '20 at 08:29
@J.Col I can't reproduce the behaviour you described in the edit... — Sweeper, Feb 20 '20 at 08:35
`Encoding.Default` can *vary* from workstation to workstation (it may be `Win-1251`, `Win-1252`, `Win-1255` etc); in case of .Net **Core** it's `UTF-8` — Dmitry Bychenko, Feb 20 '20 at 09:04

score 1 · Accepted Answer · answered Feb 20 '20 at 08:52

   byte[] utf8Bytes = new byte[text_txt.Length];
                for (int i = 0; i < text_txt.Length; ++i)
                {
                    //Debug.Assert( 0 <= utf8String[i] && utf8String[i] <= 255, "the char must be in byte's range");
                    utf8Bytes[i] = (byte)text_txt[i];
                }
                text_txt= Encoding.UTF8.GetString(utf8Bytes, 0, text_txt.Length);

from answer: How to convert a UTF-8 string into Unicode?

Dmitry Bychenko · Answer 2 · 2020-02-20T09:12:39.670

Well, you probably mean this:

// Forward: given in UTF-8 represented in WIN-1252

  byte[] data = Encoding.UTF8.GetBytes("привет");
  string text = Encoding.GetEncoding(1252).GetString(data);

// Reverse: given in WIN-1252 represented in UTF-8

  byte[] reversedData = Encoding.GetEncoding(1252).GetBytes("Ð¿Ñ€Ð¸Ð²ÐµÑ‚");
  string reversedText = Encoding.UTF8.GetString(reversedData);

  Console.WriteLine($"{string.Join(" ", data)} <=> {text}");
  Console.WriteLine(reversedText);

Outcome:

208 191 209 128 208 184 208 178 208 181 209 130 <=> Ð¿Ñ€Ð¸Ð²ÐµÑ‚
привет

Please, note that you've omitted € and , characters:

 Ð¿Ñ Ð¸Ð²ÐµÑ  - actual string
 Ð¿Ñ€Ð¸Ð²ÐµÑ‚ - should be

score -1 · Answer 3 · answered Feb 20 '20 at 08:22

-1

You need to be explicit about the type of encoding you're using to convert to bytes, (Syste.Text.Encoding.UTF8.GetBytes). eg:

using System;
using System.Text;

public class Program {
    public static void Main() {
        string text_txt = "Ð¿ÑÐ¸Ð²ÐµÑ";

        byte[] bytesUtf8 = Encoding.UTF8.GetBytes(text_txt);
        text_txt = Encoding.UTF8.GetString(bytesUtf8);

        Console.WriteLine(text_txt);
    }
}

This way UTF8 is used to both encode and decode the string the same way, and when you ensure the same string comes back from the GetString method.

answered Feb 20 '20 at 08:22

Fehr

476
4
5

Did you try it on your console? Nothing converted.. – J.Col Feb 20 '20 at 08:28
His original post converted "Ð¿ÑÐ¸Ð²ÐµÑ" to п�?иве�. I assumed this was his issue, but I'm otherwise unclear on why the poster expects the input should produce "привет" rather then the input once reversed. – Fehr Feb 20 '20 at 08:33

Encode UTF8 text to Unicode C#

3 Answers3