writing unicode characters/string to file

Question

I'm trying to write unicode characters to file with std::wofstream but the put or write function doesn't write any characters.

Sample code:

#include <fstream>
#include <iostream>

int main()
{
    std::wofstream file;
    file.open("output.txt", std::ios::app);
    if (file.is_open())
    {
        wchar_t test = L'й';
        const wchar_t* str = L"фывдлао";
        file.put(test);
        file.write(str, sizeof(str));
        file.close();
    }
    else
    {
        std::wcerr << L"Failed to open file" << std::endl;
    }

    std::cin.get();
    return 0;
}

output.txt file is empty, no wchar/string is written after executing code, why? what am I doing wrong?

EDIT: Corected code:

#include <fstream>
#include <iostream>

int main()
{
    std::wofstream file;
    file.open("output.txt", std::ios::app);
    if (file.is_open())
    {
        wchar_t test = L'й';
        const wchar_t* str = L"фывдлао";
        file.put(test);
        if (!file.good())
        {
            std::wcerr << L"Failed to write" << std::endl;
        }
        file.write(str, 8);
        file.close();
    }
    else
    {
        std::wcerr << L"Failed to open file" << std::endl;
    }

    std::cin.get();
    return 0;
}

After applying code correction I'm presented with Failed to write but I still don't understand what do I need to do to write wide strings and chars?

FWIW: Wide chars in C++ are problematic, and AFAIK wchar_t is likely to cause more problems that it solves. An alternative implementation, like QString, may be a much safer choice. — Frax, Apr 13 '19 at 17:41
QString? Surely there must be a way to work with unicode with standard libraries or native OS API's ? I just never encountered this problem with wide version of function/object that works in an unexpected way. ie. wide version of offstream deals with `char` instead of `wchar_t` what is the logic behind this? — , Apr 13 '19 at 17:49
@Frax `QString` is also based on `wchar_t`. The only advantage it has is a stable versioned ABI from exactly one source. — Deduplicator, Apr 13 '19 at 17:58
@Deduplicator For one thing, QString handles UTF-8 properly and out of the box. In general, it is portable, while wchar_t not so much, as it has different size on different platforms. I'm not sure how it work in practice. The point is, the cpp standard gives you little guarantees about what wchar_t is, while QString is quite specific (i.e. the stable ABI; it's a big deal, actually). — Frax, Apr 13 '19 at 18:48
@Frax Sorry, I was wrong. QString is based on their own UTF-16 codeunit-class, not on `wchar_t`. — Deduplicator, Apr 13 '19 at 19:02

score 4 · Answer 1 · 2019-04-14T09:53:02.253

I made it work this way, no need for external string libraries such as QString!

sole use of std libraries and c++11

#include <iostream>
#include <locale>
#include <codecvt>
#include <fstream>
#include <Windows.h>

int main()
{
    std::wofstream file;
    // locale object is responsible of deleting codecvt facet!
    std::locale loc(std::locale(), new std::codecvt_utf16<wchar_t> converter);

    file.imbue(loc);
    file.open("output.txt"); // open file as UTF16!

    if (file.is_open())
    {
        wchar_t BOM = static_cast<wchar_t>(0xFEFF);
        wchar_t test_char = L'й';
        const wchar_t* test_str = L"фывдлао";

        file.put(BOM);
        file.put(test_char);
        file.write(test_str, lstrlen(test_str));

        if (!file.good())
        {
            std::wcerr << TEXT("Failed to write") << std::endl;
        }

        file.close();
    }
    else
    {
        std::wcerr << TEXT("Failed to open file") << std::endl;
    }

    std::wcout << TEXT("Done!") << std::endl;

    std::cin.get();
    return 0;
}

File output:

йфывдлао

Note that `std::codecvt_utf16` is deprecated and has no replacement as of yet. You might consider saving the file in UTF8 format, convert back to UTF16 for WinAPI — Barmak Shemirani, Apr 22 '19 at 22:19

score 2 · Answer 2 · answered Apr 13 '19 at 17:14

2

The first problem happens immediately: put is not capable of writing wide chars and stream will fail, however you never check whether first write succeeded:

file.put(test);
if(not file.good())
{
    std::wcerr << L"Failed to write" << std::endl;
}

Second problem is that sizeof(str) returns size of the pointer in bytes, not size of the string in bytes.

answered Apr 13 '19 at 17:14

user7860670

35,849
4
58
84

Thanks, I updated my question with correction... what do I need to do to write wide strings? you said `put` doesn't accept wide chars but I see it's expecting a parameter of `wchar_t` since it's `wofstream` a wide version, so why it doesn't work? – Apr 13 '19 at 17:21
@zebanovich Even though it is a "wide" version it still operates on "single" chars. So the only way to write wide chars would be to open file in binary mode and use `write` method. – user7860670 Apr 13 '19 at 17:31
OK, I tried `std::ios::binary` and using only `write` method but it still doesn't write wide string out. fail bit is set. – Apr 13 '19 at 17:36
@zebanovich Are you sure that you've opened normal stream (not wide stream that will fail every time wide char can not be narrowed) in binary mode? – user7860670 Apr 13 '19 at 17:40
I tried that too, using normal stream that is `std::ofstream` results in wrong output. `warning C4244: 'argument': conversion from 'wchar_t' to '_Elem', possible loss of data` – Apr 13 '19 at 17:46
@zebanovich Just try `file.write(reinterpret_cast(str), wcslen(str) * sizeof(wchar_t));` – user7860670 Apr 13 '19 at 17:49
Thanks, Tried that, bad output... instead of `фывдлао` it writes `DK24;0>` to file – Apr 13 '19 at 17:54
@zebanovich Then it means that it is written correctly. Note that some text editors recognize UTF16 only when BOM is present – user7860670 Apr 13 '19 at 17:56

writing unicode characters/string to file

2 Answers2

Related