Can't write chinese character into textfile with wofstream

Question

I'm using std::wofstream to write characters in a text file.My characters can have chars from very different languages(english to chinese). I want to print my vector<wstring> into that file. If my vector contains only english characters I can print them without a problem. But if I write chineses characters my file remains empty.

I browsed trough stackoverflow and all answers said bascially to use functions from the library:

#include <codecvt>

I can't include that library, because I am using Dev-C++ in version 5.11. I did:#define UNICODE in all my header files. I guess there is a really simple solution for that problem. It would be great, if someone could help me out.

My code:

#define UNICODE
#include <string>
#include <fstream>

using namespace std;

int main()
{
    string Path = "D:\\Users\\\t\\Desktop\\korrigiert_RotCommon_zh_check_error.log";
    wofstream Out;
    wstring eng = L"hello";
    wstring chi = L"程序";

    Out.open(Path, ios::out);

    //works.
    Out << eng;

    //fails
    Out << chi;

    Out.close();

    return 0;
}

Kind Regards

Answer 1

Even if the name of the wofstream implies it's a wide char stream, it's not. It's still a char stream that uses a convert facet from a locale to convert the wchars to char.

Here is what cppreference says:

All file I/O operations performed through std::basic_fstream<CharT> use the std::codecvt<CharT, char, std::mbstate_t> facet of the locale imbued in the stream.

So you could either set the global locale to one that supports Chinese or imbue the stream. In both cases you'll get a single byte stream.

#include <locale>
//...
const std::locale loc = std::locale(std::locale(), new std::codecvt_utf8<wchar_t>);

Out.open(Path, ios::out);
Out.imbue(loc);

Unfortunately std::codecvt_utf8 is already deprecated[2]. This MSDN magazine article explains how to do UTF-8 conversion using MultiByteToWideChar C++ - Unicode Encoding Conversions with STL Strings and Win32 APIs.

Here the Microsoft/vcpkg variant of an to_utf8 conversion:

std::string to_utf8(const CWStringView w)
{
    const size_t size = WideCharToMultiByte(CP_UTF8, 0, w.c_str(), -1, nullptr, 0, nullptr, nullptr);
    std::string output;
    output.resize(size - 1);
    WideCharToMultiByte(CP_UTF8, 0, w.c_str(), -1, output.data(), size - 1, nullptr, nullptr);
    return output;
 }

On the other side you can use normal binary stream and write the wstring data with write().

std::ofstream Out(Path, ios::out | ios::binary);

const uint16_t bom = 0xFEFF;
Out.write(reinterpret_cast<const char*>(&bom), sizeof(bom));    // optional Byte order mark

Out.write(reinterpret_cast<const char*>(chi.data()), chi.size() * sizeof(wchar_t));

Answer 2

You forgot to tell your stream what locale to use:

Out.imbue(std::locale("zh_CN.UTF-8"));

You'll obviously need to include <locale> for this.