Unexpected results with wchar_t and c_str string conversion

Question

Based on this answer to a related question, I tried to write a method that converts a standard string to a wide string, which I can then convert into a wchar_t*.

Why aren't the two different ways of creating the wchar_t* equivalent? (I've shown the values that my debugger gives me).

TEST_METHOD(TestingAssertsWithGetWideString)
{
   std::wstring wString1 = GetWideString("me");
   const wchar_t* wchar1 = wString1.c_str(); // wchar1 = "me"
   const wchar_t* wchar2 = GetWideString("me").c_str(); // wchar2 = "ﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮ@" (Why?!)
}

where GetWideString is defined as follows:

inline const std::wstring GetWideString(const std::string &str)
{
   std::wstring wstr;
   wstr.assign(str.begin(), str.end());

   return wstr;
};

Note: the following doesn't work either.

const wchar_t* wchar2 = GetWChar("me");

const wchar_t *GetWChar(const std::string &str)
{
   std::wstring wstr;
   wstr.assign(str.begin(), str.end());

   return wstr.c_str();
};

`// Why doesn't this work?!` - You have a pointer to a temporary buffer. — chris, Oct 08 '13 at 00:13

Remy Lebeau · Accepted Answer · 2013-10-08T16:36:27.033

2

Each time you call GetWideString(), you are creating a new std::wstring, which has a newly allocated memory buffer. You are comparing pointers to different memory blocks (assuming Assert::AreEqual() is simply comparing the pointers themselves and not the contents of the memory blocks that are being pointed at).

Update: const wchar_t* wchar2 = GetWideString("me").c_str(); does not work because GetWideString() returns a temporary std::wstring that goes out of scope and gets freed as soon as the statement is finished. Thus you are obtaining a pointer to a temporary memory block, and then leaving that pointer dangling when that memory gets freed before you can use the pointer for anything.

Also, const wchar_t* wchar2 = GetWChar("me"); should not compile. GetWChar() returns a std::wstring, which does not implement an implicit conversion to wchar_t*. You have to use the c_str() method to get a wchar_t* from a std::wstring.

edited Oct 08 '13 at 16:36

answered Oct 08 '13 at 00:17

Remy Lebeau

555,201
31
458
770

Good point, but I've edited my question to clarify what I'm looking for. – sgryzko Oct 08 '13 at 15:53
Thanks for the explanation. I didn't realize that the std::wstring returned was going out of scope. (PS I've fixed the part that you said shouldn't compile. That was a typo). – sgryzko Oct 08 '13 at 18:25
Am I do understand that `std::string` uses `char`? As an old-time programmer trying to learn the new way of doing things, I'm having enough trouble finding justification for learning `std:string`. Don't tell me it doesn't support wide characters out of the box. – Jonathan Wood Mar 19 '17 at 19:50
@JonathanWood `std::string` uses `char`, yes. `std::wstring` uses `wchar_t` instead. And in C++11, there is also now `std::u16string` that uses `char16_t`, and `std::u32string` that uses `char32_t`. – Remy Lebeau Mar 19 '17 at 21:18

score 1 · Answer 2 · answered Oct 08 '13 at 00:15

1

Because the two pointers aren't equal. A wchar_t * is not a String, so you get the generic AreEqual.

answered Oct 08 '13 at 00:15

David Schwartz

179,497
17
214
278

Good point, but I've edited my question to clarify what I'm looking for. – sgryzko Oct 08 '13 at 15:52

score 0 · Answer 3 · answered Oct 08 '13 at 00:22

std::wstring contains of wide characters of type wchar_t. std::string contains characters of type char. For special characters stored within std::string a multi-byte encoding is being used, i.e. some characters are represented by 2 characters within such a string. Converting between these thus can not be easy as calling a simple assign.

To convert between "wide" strings and multi-byte strings, you can use following helpers (Windows only):

// multi byte to wide char:
std::wstring s2ws(const std::string& str)
{
    int size_needed = MultiByteToWideChar(CP_UTF8, 0, &str[0], (int)str.size(), NULL, 0);
    std::wstring wstrTo(size_needed, 0);
    MultiByteToWideChar(CP_UTF8, 0, &str[0], (int)str.size(), &wstrTo[0], size_needed);
    return wstrTo;
}

// wide char to multi byte:
std::string ws2s(const std::wstring& wstr)
{
    int size_needed = WideCharToMultiByte(CP_ACP, 0, wstr.c_str(), int(wstr.length() + 1), 0, 0, 0, 0); 
    std::string strTo(size_needed, 0);
    WideCharToMultiByte(CP_ACP, 0, wstr.c_str(), int(wstr.length() + 1), &strTo[0], size_needed, 0, 0); 
    return strTo;
}

Is it correct to assume that without any special characters, my simple assign will work as expected? — sgryzko, Oct 08 '13 at 15:55

Unexpected results with wchar_t and c_str string conversion

3 Answers3