c_str() produces different output when string is originally in a std::vector

Question

I am trying to token a string and I'm getting some strange behaviour. Here is the code:

ifstream fin("myFile.txt");
char* in[256];
fin.getline(in,256);
string s = in;
vector<string> f;
f.push_back(s);
char* pch1,pch2;
pch1 = strtok((char*)s.c_str()," ");
pch1 = strtok(NULL," ");
pch2 = strtok((char*)f[0].c_str()," ");
pch2 = strtok(NULL," ");

if the file contains the line "This is a string", then the value stored at location pch1 will be "is", but pch2 will point to null. Why is that?

I printed out the bytes of the two strings, s and f[0]. s contains all the characters I expect, including the space. However, f[0] contains the 0 character instead of spaces. I understand that sometimes istreams can treat the space as a null character, but shouldn't s and f[0] contain exactly the same bytes?

Don't use `strtok()` please when working with `std::string`. — πάντα ῥεῖ, Feb 17 '16 at 15:14
What is this unholy mishmash of languages? Why use any ancient, often unsafe C functions when C++ goes to great pains to provide you with an extremely comprehensive and typesafe standard library? That's missing the point entirely. Please, pick one language XOR the other. — underscore_d, Feb 17 '16 at 15:17
Here's a solution to your problem, Michael: http://stackoverflow.com/questions/10051679/c-tokenize-string . — Topological Sort, Feb 17 '16 at 15:22
This code won't compile. You have declare `in` as an array of character pointers, and then written `string s = in`. `in` will decay to `char**` and `std::string` has no constructor which takes that. — Martin Bonner supports Monica, Feb 17 '16 at 15:36
`stream`s do not treat space as a null character. Extractors stop extracting when they encounter whitespace. — Pete Becker, Feb 17 '16 at 16:40
Thanks for the answer and the advice. I still have a lot to learn, I guess. — Michael Stachowsky, Feb 17 '16 at 19:08

score 5 · Answer 1 · answered Feb 17 '16 at 15:18

5

strtok modifies the string, it replaces occurrences of the delimiter with '\0'. c_str() returns a const char*, you should not simply cast the const away.

answered Feb 17 '16 at 15:18

alain

11,939
2
31
51

3

Yup, in all cases, modifying a `const` variable via non-`const` reference/pointer results in UB. That's probably just one of the problems with this attempt. – underscore_d Feb 17 '16 at 15:24

Edward Strange · Answer 2 · 2016-02-17T16:38:04.317

2

You are causing all kinds of undefined behavior. I would recommend staying away from stroke and instead look into boost's tokenizer library. Your brain cells will appreciate it. Just forget that the entire C library exists in C++ unless you really, really, really need it to talk to another C library.

edited Feb 17 '16 at 16:38

answered Feb 17 '16 at 15:33

Edward Strange

40,307
7
73
125

c_str() produces different output when string is originally in a std::vector

2 Answers2