String tokenization code gets stuck in while loop

Question

The task was to find all the words in array s with numbers inside them. The word are spaced with " " and at the end of the array there is always a ..

My code doesn't exit the while loop and I don't know why:

#include <iostream>
#include <vector>
#include <cstdio>
#include <cstring>
#include <cctype>

using namespace std;
void Numbers(char s[], vector<pair<int, char>>& nums){
    char *token;
    int zero = 0;
    token = strtok(s, " ");
    for(int i = 0; i < strlen(token); i++){
        if(isdigit(token[i])){
            nums.emplace_back(zero, *token);
        }
    }
    while(token != NULL){
        zero ++;
        token = strtok(NULL, " ");
        for(int i = 0; i < strlen(token); i++){
            if(isdigit(token[i])){
                nums.emplace_back(zero, *token);
            }
        }
    }
}
int main() {
    vector<pair<int, char>> nums;
    char s[256];
    gets(s);
    Numbers(s, nums);
    cout << nums.size();
    for (int i = 0; i < nums.size(); i ++){
        cout << nums[i].first << " " << nums[i].second << endl;
    }
    return 0;
}

I tried to run the code with qw6wqe esd9f 0gfgkj. input with debug, token changes, all should be working, but it never gets out of the loop.

Tried using break but that didn't work.

@МихайлоГорак See: [How do I tokenize a string in C++?](https://stackoverflow.com/questions/53849/how-do-i-tokenize-a-string-in-c) — digito_evo, May 23 '23 at 15:59
@МихайлоГорак [See this](https://godbolt.org/z/e3Yz9are9) — PaulMcKenzie, May 23 '23 at 16:16

score 2 · Accepted Answer · answered May 23 '23 at 16:13

2

Your program has undefined behaviour. When you reach the end of the input, token is a null pointer, and you then pass it to strlen.

You need to stop processing when strtok returns a null pointer.

void Numbers(char s[], vector<pair<int, char>>& nums){
    int count = 0;
    for(char * token = strtok(s, " "); token = strtok(nullptr, " ");){
        for(int i = 0; i < strlen(token); i++){
            if(isdigit(token[i])){
                nums.emplace_back(count, *token);
            }
        }
        count++;
    }
}

answered May 23 '23 at 16:13

Caleth

52,200
2
44
75

1

`nums.emplace_back(count, *token)` should presumably be `nums.emplace_back(count, token[i])` – Alan Birtles May 23 '23 at 16:48
@AlanBirtles Perhaps, or it might want to be filling `vector> nums` – Caleth May 24 '23 at 08:23

String tokenization code gets stuck in while loop

1 Answers1