strtok(): convert string to char*

Question

Possible Duplicate:
Splitting a string in C++

char *strtok(char *s1, const char *s2)

How can I convert a string to a char* as required by strtok? I did

for (string line; getline(sourceFile, line);) {
    tokens = strtok(line.c_str(), " {};");
}

Where sourceFile is an ifstream (sourceFile.open(filepath.c_str());)

I am getting:

argument of type "const char *" is incompatible with parameter of type "char *"

Since it's C++, how about this? http://stackoverflow.com/questions/236129/splitting-a-string-in-c — chris, Oct 23 '12 at 00:11
You don't want to use `strtok()` on a C++ string; `strtok()` mangles the string, inserting NUL `'\0'` at token ends. Use something — anything — else. — Jonathan Leffler, Oct 23 '12 at 00:12

score 2 · Answer 1 · answered Oct 23 '12 at 00:19

2

As others have said, you probably want to use something other than strtok.

However, to do what you are asking (and you probably shouldn't):

for (string line; getline(sourceFile, line);) {
    char* line_cstr = strdup(line.c_str());

    char* token = strtok(line_cstr, " {};");

    while ((token = strtok(NULL, " {};")) != NULL) {
        //code
    }

    free(line_cstr);

}

answered Oct 23 '12 at 00:19

Geoff Montee

2,587
13
14

1

That is going to work well at the expense of duplicating the string, but I would recommend simply using boost::tokenizer instead. – Timo Geusch Oct 23 '12 at 00:27
@TimoGeusch I completely agree. No need to use these messy C functions when better alternatives are available. – Geoff Montee Oct 23 '12 at 00:28

score 0 · Answer 2 · answered Oct 23 '12 at 00:17

0

If you do something like this:

char fp[40];  // or some reasonable size
strcpy( fp, filepath.c_str()); // fp is not a const char* like c_str() returns

You should be able to use strtok() against fp.

But you might want to consider a more C++ way of splitting strings.

answered Oct 23 '12 at 00:17

Chimera

5,884
7
49
81

1

Even then, making `fp` a `std::vector(filepath.size())` would be better. – chris Oct 23 '12 at 00:18
1

@chris: `std::vector(filepath.size())` - it should be `size() + 1`, because you'll need it NUL terminated for `strtok()`. – Tony Delroy Oct 23 '12 at 00:27

Tony Delroy · Answer 3 · 2012-10-23T00:31:29.047

As Jonathan Leffler's comment explains, strtok actually modifies the buffer in place to terminate successive tokens. That's great for speed - saves copying each token out to some other memory area while still giving you convenient access to it as a separate NUL-terminated sequence. It's an interesting question whether it's legitimate to use strtok on a std::string instance - certainly, the c_str() member returns a const char* because you're not meant to write over parts of that buffer. But, as long as you don't have an empty string (for which it'd give undefined behaviour), you can get a char* to the first element with &line[0]. Having done that, you can mutate the string, but only at offsets [0]..[size()-1]... it's not guaranteed to be NUL terminated the way the c_str()-returned buffer is, and precisely because of that the &line[0]-returned buffer may not be suitable for use by strtok. You could append a NUL (i.e. line += '\0' - NUL is legal std::string content), but it would make the code a bit hackish and harder to maintain, and you might need to remove the NUL afterwards if you're planning to use the original value for anything.

If you really want to use strtok, seems best to copy the data to a separate writable NUL-terminated buffer first. For example:

char* p = strdup(line.c_str());
... use strtok on p ...
free(p);

You could use alloca(), a smart pointer, a std::vector<char> to minimise potential for memory leaks. (I don't see any particular reason to prefer C++ new and delete for the above - you're using strtok anyway, but if you've smart-pointer libraries that expect it go for it).

All that said, finding another mechanism - like those in boost - is a better option. I'm guessing alternatives like that are discussed in the link chris posted in a comment under your question....

strtok(): convert string to char*

3 Answers3