Convert in Another Format With Character Encoding

Question

I am using Oracle 10g database for a C# application. The issue is, in the database, the NVARCHAR column doesn't save other languages except English. As NVARCHAR supports Unicode, this should be working. But instead I've tried a simple method using a tutorial as follows:

Encoding ascii = Encoding.ASCII;
Encoding unicode = Encoding.Unicode;

//Convert the string into a byte[].
byte[] unicodeBytes = ascii.GetBytes("আমার সোনার বাংলা!"); //Text to show

//Perform the conversion from one encoding to the other.
byte[] asciiBytes = Encoding.Convert(ascii, unicode, unicodeBytes);
char[] asciiChars = new char[ascii.GetCharCount(asciiBytes, 0, asciiBytes.Length)];
ascii.GetChars(asciiBytes, 0, asciiBytes.Length, asciiChars, 0);
string asciiString = new string(asciiChars);
Console.WriteLine(asciiString);

Console.ReadKey();

May seems silly but was expecting if I can show the text in the console app with the above format. Now it shows question (??????) marks. Any way I can show the text and save it at least in any other format, so I can retrieve and show it appropriately in the front-end.

Answer 1

If you can use unicode (and you should, hey its 2018), then it would be best to avoid Bijoy altogether. Process and store everything that is a string, as System.String in .NET and as NVARCHAR in Oracle.

The Windows console can handle unicode without any problems, if we observe two important prerequisites that the documentation clearly states:

Support for Unicode [...] requires a font that has the glyphs needed to render that character. To successfully display Unicode characters to the console, the console font must be set to a [...] font such as Consolas or Lucida Console

This is something you must ensure in Windows setting, independently from your .NET application.

The second prerequisite, emphasis mine:

[...] Console class supports UTF-8 encoding [...] Beginning with the .NET Framework 4.5, the Console class also supports UTF-16 encoding [...] To display Unicode characters to the console. you set the OutputEncoding property to either UTF8Encoding or UnicodeEncoding.

What the documentation does not say, is that none of the fonts that can be selected from the properties menu of the console window will normally contain glyphs of all alphabets in the world. If you needed right-to-left capability as for example with Hebrew or Arabic, you're out of luck.

If the program is running a Windows version without the east asian fonts preinstalled, follow this tutorial to install the Bangla LanguageInterfacePack (KB3180030).

Then apply this answer to our problem as follows:

1. open the windows registry editor
2. navigate to HKLM\Software\Microsoft\WindowsNT\CurrentVersion\Console\TrueTypeFont
3. create a new string value, assign an available key like "000", and the value "Bangla Medium"
4. reboot the PC

Now set the console font to "Bangla", using the window menu of the console, last menu item "Properties", second tab "Font".

Finally get rid of all that encoding back and forth, and simply write:

using System;
using System.Text;

namespace so49851713
{
    class Program
    {
        public static void Main()
        {
            var mbb = "\u263Aআমার সোনার বাংলা!";
            /* prepare console (once per process) */
            Console.OutputEncoding = UTF8Encoding.UTF8;
            Console.WriteLine(mbb);
            Console.ReadLine();
        }
    }
}