Conversion from char16_t(utf16) to utf8 doesn't work in visual studio 2015 and 2017

Question

I want to convert the data type char16_t to utf8, and I have written the following code to do that:

// utf16 String
char16_t *wchs = u"UTF16 string";
// A converter object
wstring_convert<codecvt_utf8_utf16<char16_t>, char16_t> conv;
// Convert from utf16 to utf8
string str = conv.to_bytes(wchs);
// print the UTF-16 string after converting to UTF-8
cout << str << endl;

The previous code doesn't work in visual studio whether 2015 or 2017, specifically with "char16_t" data type and gives me an error:
error LNK2001: unresolved external symbol "public: static class std::locale::id std::codecvt<char16_t,char,struct _Mbstatet>::id" (?id@?$codecvt@_SDU_Mbstatet@@@std@@2V0locale@2@A),
but works well in the GCC compiler.

Why that happened and how to solve that problem?

https://stackoverflow.com/questions/42946335/deprecated-header-codecvt-replacement — Hans Passant, Apr 23 '20 at 22:16

score 2 · Answer 1 · answered Apr 23 '20 at 22:10

2

You might be out of luck as this seems to be a Visual Studio bug since there are at least 3 posts on the VS Developer Community page which describe the given problem

where all of them state that it has been

fixed in: visual studio 2019 version 16.2

with no indication of a backport to VS 2015 or even VS 2017.

answered Apr 23 '20 at 22:10

Pixelchemist

24,090
7
47
71

So, the only alternative way for conversion in vs2015 or 2017 is `wchar_t` instead `char16_t`? – Lion King Apr 23 '20 at 22:30
Probably. They are both used for UTF-16 on Windows, so a simple type-cast should suffice: `const char16_t *wchs = u"UTF16 string"; wstring_convert, wchar_T> conv; string str = conv.to_bytes(reinterpret_cast(wchs));` Use an `#ifdef` if you need to compile the code for other platforms. – Remy Lebeau Apr 23 '20 at 23:40

Rasit Simsek · Answer 2 · 2021-01-27T13:57:35.657

Here is a simple function how you can convert from std::u16string to std::string. It works with VS 2017.

std::string utf16_to_utf8( std::u16string&& utf16_string )
{
   std::wstring_convert<std::codecvt_utf8_utf16<int16_t>, int16_t> convert;
   auto p = reinterpret_cast<const int16_t *>( utf16_string.data() );
   return convert.to_bytes( p, p + utf16_string.size() );
}

The reason is the declaration of wstring_convert<codecvt_utf8_utf16<char16_t>, char16_t> conv; with char16_t template parameter does not compile under VS 2015 and VS2017 due of missing initialization of static locale::id id; in a .cpp file. The int16_t type is working.

Under Linux with gcc or clang compiler suites it is a little bit easier to convert from std::u16string. There is no issue with static locale::id id;.

std::string utf16_to_utf8( std::u16string&& utf16_string )
{
   std::wstring_convert<std::codecvt_utf8_utf16<char16_t>, char16_t> convert;
   return convert.to_bytes( utf16_string );
}

it doesn't answer the question of why that happened though, can you please elaborate why your solution is working and topic starters' not? — Antigluk, Jan 27 '21 at 09:31

Conversion from char16_t(utf16) to utf8 doesn't work in visual studio 2015 and 2017

2 Answers2