36

This code compiled OK under VS2013:

std::string Unicode::utf16_to_utf8(std::u16string utf16_string)
{
    std::wstring_convert<std::codecvt_utf8_utf16<char16_t>, char16_t> convert;
    return convert.to_bytes(utf16_string);
}

Now with VS2015 I get:

1>unicode.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: static class std::locale::id std::codecvt<char16_t,char,struct _Mbstatet>::id" (__imp_?id@?$codecvt@_SDU_Mbstatet@@@std@@2V0locale@2@A)
user3443139
  • 654
  • 1
  • 7
  • 11

4 Answers4

37

Old question, but for future reference: this is a known bug in Visual Studio 2015, as explained in the latest post (January 7th 2016) in this thread of MSDN Social.

The workaround for your example looks like this (I implemented your method as a free function for simplicity):

#include <codecvt>
#include <locale>
#include <string>
#include <iostream>

#if _MSC_VER >= 1900

std::string utf16_to_utf8(std::u16string utf16_string)
{
    std::wstring_convert<std::codecvt_utf8_utf16<int16_t>, int16_t> convert;
    auto p = reinterpret_cast<const int16_t *>(utf16_string.data());
    return convert.to_bytes(p, p + utf16_string.size());
}

#else

std::string utf16_to_utf8(std::u16string utf16_string)
{
    std::wstring_convert<std::codecvt_utf8_utf16<char16_t>, char16_t> convert;
    return convert.to_bytes(utf16_string);
}

#endif

int main()
{
    std::cout << utf16_to_utf8(u"Élémentaire, mon cher Watson!") << std::endl;

    return 0;
}

Hopefully, the problem will be fixed in future releases, otherwise the #if condition will need refining. UPDATE: nope, not fixed in VS 2017. Therefore, I've updated the preprocessor conditional to >= 1900 (initially was == 1900).

JPNotADragon
  • 2,050
  • 2
  • 25
  • 31
  • 7
    Well, such an approach from M$ developers is depressing... Imagine you have tons of code you need to rewrite in such a manner... – Danatela Jun 15 '16 at 10:56
  • 2
    That worked for me. How about the opposite direction? If I do an auto p = reinterpret_cast(utf8_string.data()); and convert.from_bytes(p) instead, I get that "no suitable user-defined conversion from "std::basic_string, std::allocator>" to "std::u16string" exists" – marc40000 Jun 02 '18 at 00:50
  • found a similar solution and it's reverse at an antlr fix. but not optimal because of two conversions : https://github.com/antlr/antlr4/commit/de6f04be0beebd17ea6232f554635c5262127aaa – Mohan Kumar Jan 31 '19 at 13:42
  • 4
    still not fixed in VS 2019 Preview. – Mohan Kumar Jan 31 '19 at 13:42
9

Define the missing symbol in a cpp file.

// Apparently Microsoft forgot to define a symbol for codecvt.
// Works with /MT only
#include <locale>

#if (!_DLL) && (_MSC_VER >= 1900 /* VS 2015*/) && (_MSC_VER <= 1911 /* VS 2017 */)
std::locale::id std::codecvt<char16_t, char, _Mbstatet>::id;
#endif
pascalx
  • 101
  • 2
  • 3
  • 2
    VS2017: error C2491: 'std::codecvt::id': definition of dllimport static data member not allowed – Bogdan Sep 27 '17 at 20:51
  • 2
    Indeed it does not compile with **/MD**, because the foreign DLL would have no way of knowing/using our definition. It does work with **/MT** though. So it is only half a workaround :-) – pascalx Sep 27 '17 at 22:10
8

This worked for me in VS2017:

std::wstring utf8_to_utf16(std::string utf8_string)
{
   return std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>, wchar_t>{}.from_bytes(utf8_string);
}

std::string utf16_to_utf8(std::wstring utf16_string)
{
    return std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>, wchar_t>{}.to_bytes(utf16_string);
}
Paul Williams
  • 3,099
  • 38
  • 34
4

Another possible workaround is to use default second template parameter (wchar_t) for wstring_convert. It is working for "MS Visual Studio 2015 update 3". Please note that it is not platform-independent solution. Windows only.

std::string utf16_to_utf8(std::u16string u16_string)
{
    std::wstring wide_string(u16_string.begin(), u16_string.end());
    std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> convert;
    return convert.to_bytes(wide_string);
}
aleksandrm8
  • 320
  • 2
  • 5