strtok_r for MinGW

Question

strtok_r is the reentrant variant of strtok. It is POSIX-conformant. However, it is missing from MinGW, and I'm trying to compile a program that is using it.

Is there any way I could add a standard implementation of this function, perhaps to the project's own code, or to MinGW's standard library functions?

score 15 · Accepted Answer · answered Oct 19 '12 at 17:23

Since there are some license questions about the code from another answer, here's one that's explicitly public domain:

/* 
 * public domain strtok_r() by Charlie Gordon
 *
 *   from comp.lang.c  9/14/2007
 *
 *      http://groups.google.com/group/comp.lang.c/msg/2ab1ecbb86646684
 *
 *     (Declaration that it's public domain):
 *      http://groups.google.com/group/comp.lang.c/msg/7c7b39328fefab9c
 */

char* strtok_r(
    char *str, 
    const char *delim, 
    char **nextp)
{
    char *ret;

    if (str == NULL)
    {
        str = *nextp;
    }

    str += strspn(str, delim);

    if (*str == '\0')
    {
        return NULL;
    }

    ret = str;

    str += strcspn(str, delim);

    if (*str)
    {
        *str++ = '\0';
    }

    *nextp = str;

    return ret;
}

score 2 · Answer 2 · answered Oct 19 '12 at 13:20

2

Here's the source code which you can simply add to your own library/function in your project:

char *strtok_r(char *str, const char *delim, char **save)
{
    char *res, *last;

    if( !save )
        return strtok(str, delim);
    if( !str && !(str = *save) )
        return NULL;
    last = str + strlen(str);
    if( (*save = res = strtok(str, delim)) )
    {
        *save += strlen(res);
        if( *save < last )
            (*save)++;
        else
            *save = NULL;
    }
    return res;
}

answered Oct 19 '12 at 13:20

P.P

117,907
20
175
238

This is the standard implementation of strtok_r. If `str` is a on heap then this is thread-safe. – P.P Oct 19 '12 at 13:35
Is it reasonable to expect str to be on heap? Meaning, is the requirement part of the spec? – sashoalm Oct 19 '12 at 13:41
1

Is this your own local implementation or did it come from an open source project? If its the latter, is there any license associated with its use? – simonc Oct 19 '12 at 13:58
libc source. This is very simple. Anyone can implement without ever looking at the source. I don't think there's any license associated with this. – P.P Oct 19 '12 at 14:05
Sorry if I'm being dim but do you have a link to the source? It seems unlikely to me that it'll have been released without any license but I expect I'll be proved wrong :-) – simonc Oct 19 '12 at 14:10
AFAIK `strtok` stores a statically allocated pointer to the current token. Every time you call `strtok` passing a non-null `delim` you are initializing this pointer, so you may affect another thread that may use `strtok` at the same time... From the OpenGroup specification: "*The strtok() function need not be reentrant. A function that is not required to be reentrant is not required to be thread-safe.*" (http://pubs.opengroup.org/onlinepubs/009695399/functions/strtok_r.html). Excuse me if I'm wrong. – Claudi Oct 19 '12 at 18:13
(Continue) Actually, the implementation I posted in my answer does not use `strtok`. Actually, `strtok` is a particular case `strtok_r` that uses the statically allocated pointer I commented. – Claudi Oct 19 '12 at 18:14

score 1 · Answer 3 · answered Oct 19 '12 at 13:21

1

Is the FreeBSD implementation any use to you?

Its liberally licensed but integrating it may have some requirements on your project documentation (adding an acknowledgement that the code has been included).

answered Oct 19 '12 at 13:21

simonc

41,632
12
85
103

score 1 · Answer 4 · answered Oct 19 '12 at 13:27

1

MINGW has no implementation of strtok_r. However you can find a thread-safe implementation in the link below:

http://www.raspberryginger.com/jbailey/minix/html/strtok__r_8c-source.html

answered Oct 19 '12 at 13:27

Claudi

5,224
17
30

strtok_r for MinGW

4 Answers4

Linked