How to display L"أَبْجَدِيَّة عَرَبِيَّة‎中文" using wcout?

Question

I want to display an Arabic message mixed with Chinese using wcout.

The following code is OK:

#include <iostream>

using namespace std;

int main()
{
    wcout.imbue(locale("chs"));
    wcout << L"中文"; // OK
}

However, the following code doesn't work:

#include <iostream>

using namespace std;

int main()
{
    wcout.imbue(locale(/* What to place here ??? */));
    wcout << L"أَبْجَدِيَّة عَرَبِيَّة‎中文"; // Output nothing. VC++ 2012 on Win7 x64
    // Why does the main advantage of unicode not apply here?
}

I think the concept of code pages should be deprecated after the adoption of unicode.

Q1. What's the mechanism of wout's displaying such a text?

Q2. Why does Windows, as a unicode-based OS, not support outputting unicode characters in its console window?

Answer 1

CRT would treat all output to files as ANSI by default. You can change that with this line at the start of your program

_setmode(_fileno(stdout), _O_WTEXT);

A good reference @ http://www.siao2.com/2008/03/18/8306597.aspx

Just for reference bidirectional language support is limited in most command prompts and from what I understand that is the limitation causing this issue here. The why it is not/supported is something that I cannot answer.

Answer 2

You cannot portably print wide strings using standard C++ facilities.

Instead you can use the open-source {fmt} library to portably print Unicode text. For example (https://godbolt.org/z/nccb6j):

#include <fmt/core.h>

int main() {
  fmt::print("أَبْجَدِيَّة عَرَبِيَّة‎中文");
}

prints

أَبْجَدِيَّة عَرَبِيَّة‎中文

This requires compiling with the /utf-8 compiler option in MSVC.

For comparison, writing to wcout on Linux (https://godbolt.org/z/h9WKsY):

std::wcout << L"أَبْجَدِيَّة عَرَبِيَّة‎中文";

prints

???????????? ?????????????

unless you switch the global locale to e.g. en_US.utf8. Similar issue exists on Windows with no standard way to fix it (you have to use non-standard CRT functions or Windows API).

Disclaimer: I'm the author of {fmt}.

Answer 3

I just read this article

"To the summary...

If you use Visual C++ you can't use UTF-8 to print text to std::cout.

If you still want to, please read this amazingly long article about how to make wcout and cout working, but it does not really give a simple solution - finally falling to redefinition of the stream buffers..." http://alfps.wordpress.com/2011/12/08/unicode-part-2-utf-8-stream-mode/

(from this blog http://blog.cppcms.com/post/105)

Answer 4

You can try this:

I assume that you were able to render Chinese only text. That signifies that you have chinese font files.

You please try with arabic only text. If you are able to render, that signifies that you have arabic font in your system.

But when you mix this, arabic + chinese, then you need to force to pick a font file which has both glyph sets. I think the default font file picked up by wcout doesnt have the arabic glyphs.

I assume that you may be getting boxes for arabic unicodes.

Answer 5

#include <iostream>
#include <io.h>
#include <fcntl.h>

int main() {
    _setmode(_fileno(stdout), _O_U16TEXT); // or _O_WTEXT
    std::wcout << L"أَبْجَدِيَّة عَرَبِيَّة‎中文" << std::endl;
}

http://www.cplusplus.com/forum/beginner/126557/

Answer 6

On Windows

I recommend redirecting the wcout buffer to a file to facilitate viewing the results, because the Windows command prompt unable to display some unicode fonts.

#include <iostream>
#include <fstream>                                                                                                                     

int main()
{
    std::locale myloc("en_US.UTF-8");
    std::locale::global(myloc);                                              

    std::wfilebuf wfbuf;
    wfbuf.open("result.txt", std::ios::out);
    std::wcout.rdbuf(std::addressof(wfbuf));   
                                                
    std::wcout << L"أَبْجَدِيَّة عَرَبِيَّة‎中文";   
 
    return 0;
}

On Linux

Method1

#include <iostream>                                                                                                                        
                       
int main()
{
    std::ios::sync_with_stdio(false);// make wcout no longer depend on stdio
    std::locale myloc("en_US.UTF-8");                                           
    std::wcout.imbue(myloc);                                                    
    std::wcout << L"أَبْجَدِيَّة عَرَبِيَّة‎中文";   
 
    return 0;
}

Method2

#include <cstdio>                                                                                                                        
                       
int main()
{
    std::locale myloc("en_US.UTF-8");                                           
    std::locale::global(myloc);// can affect stdio's locale                                               
    wprintf(L"أَبْجَدِيَّة عَرَبِيَّة‎中文");  
 
    return 0;
}

Method3

#include <cstdio>                                                                                                                        
                       
int main()
{
    std::locale myloc("en_US.UTF-8");                                           
    std::locale::global(myloc);// can affect stdio's locale                                                                 
    std::wcout << L"أَبْجَدِيَّة عَرَبِيَّة‎中文";// wcout depend on stdio, it doesn't matter if wcout's locale still C locale.

    return 0;
}