25

What's the difference between strtok and strtok_r in C and when are we supposed to use which?

Md. Sabbir Ahmed
  • 850
  • 8
  • 22
dreamer_999
  • 1,465
  • 3
  • 17
  • 22
  • 1
    One is reentrant the other is not. I think the manual states that. – RedX Mar 05 '14 at 22:18
  • 2
    Not standard C, it is Posix. Strtok() is a bug factory, it uses a global variable inside the CRT to keep track of the string position. So you can't use it on multiple strings simultaneously. Also risky in threads, most CRTs do solve that. – Hans Passant Mar 05 '14 at 22:20
  • To confuse you even more, C11 adds `strtok_s`. – Per Johansson Mar 05 '14 at 22:25

4 Answers4

28

strtok is equivalent to (and often defined as):

char *strtok(char *str, const char *delim) {
    static char *save;
    return strtok_r(str, delim, &save);
}

in general, you should use strtok_r directly rather than strtok, unless you need to make your code portable to pre-POSIX-2001 systems that only support strtok

Chris Dodd
  • 119,907
  • 13
  • 134
  • 226
21

The _r versions of functions are reentrant: you can call them from multiple threads simultaneously, or in nested loops, et cetera. Reentrant versions usually take an extra argument, this argument is used to store state between calls instead of using a global variable.

The non-reentrant versions often use global state, so if you call them from multiple threads, you are probably invoking undefined behavior. Your program could crash, or worse.

From the man pages (man 3 strtok):

The strtok_r() function is a reentrant version of strtok(). The context pointer last must be provided on each call. The strtok_r() function may also be used to nest two parsing loops within one another, as long as separate context pointers are used.

Dietrich Epp
  • 205,541
  • 37
  • 345
  • 415
  • Is there a list of such _r functions vs the original ones I may have used without knowing ? Since the _r are not standard C I can't just grep for them in *.h – dargaud Mar 08 '18 at 11:23
  • @dargaud: They *are* in the `*.h` files. The functions are not “standard C” but they are part of the POSIX standard. The only other one I can think of is `strerror_r`. – Dietrich Epp Mar 08 '18 at 16:45
5

strtok save static pointer for reuse in the next time, when you give NULL as the first parameter, so you just can't parse 2 strings in parallel.

In the strtok_r you give also the pointer, as out parameter (pointer to pointer). so there is no static pointer in the function and you can move from one string to another and back...

SHR
  • 7,940
  • 9
  • 38
  • 57
0

According the documentation, the strtok_r() function is a reentrant version of strtok(). 

char *strtok_r(char *s1, const char *s2, char **s3);

It gets the next token from string s1, where tokens are strings separated by characters from s2. To get the first token from s1, strtok_r() is called with s1 as its first parameter. Remaining tokens from s1 are obtained by calling strtok_r() with a null pointer for the first parameter. The string of delimiters, s2, can differ from call to call.

Miguel Prz
  • 13,718
  • 29
  • 42