Tokenize string without including the NULL terminator?

Question

I have a csv file that I am parsing with entries that look like:

GET,/mic-check/one.php/two/,NULL,0
POST,/mic-check/one.php,?wc-ajax=add_to_cart,0
...
GET,/mic-check/one.php/checkout/,NULL,0
GET,/mic-check/one.php/my-account/,NULL,0\0

I go through this file row by row and split each, putting the values into their respective variables. It works okay up until the absolute last line where it keeps the terminating character '%'. Example:

Method: GET
URL: /mic-check/one.php/my-account/
Query: NULL
Servers: 0%

What I would like to do, is not include this character during the parsing. Here is my code below:

int main()
{
    FILE *file;
    char row[MAX_LINE];
    char method[MAX_COLUMN];
    char url[MAX_COLUMN];
    char query[MAX_COLUMN];
    char servers[MAX_COLUMN];
    char *tkn;

    file = fopen("whitelist.csv", "r");

    while(feof(file) != true) {
        // Receive row
        fgets(row, MAX_LINE, file);
        
        // Parse row
        tkn = strtok(row, ",");
        strcpy(method, tkn);
        tkn = strtok(NULL, ",");
        strcpy(url, tkn);
        tkn = strtok(NULL, ",");
        strcpy(query, tkn);
        tkn = strtok(NULL, ",");
        strcpy(servers, tkn);

        // Use the variables
    }

    return 0;
}

"*it keeps the terminating character '%'*" Where is that character in the file? `%` is not a terminating NUL character. A text file should never contain `\0` as that would make it binary file. Are you sure it has that? Perhaps you are misinterpreting the results. Please provide complete code as a [mre] that can show the exact problem. The duplicate candidate is more likely the problem (it is definetely *a* problem). — kaylum, Jun 29 '22 at 20:42

score 1 · Accepted Answer · answered Jun 29 '22 at 20:44

Classic reading mistake.

The last read from a stream reads "upto" but not past the end of file. Thus feof() is false even though there is no data in the file. A subsequent read will then fail.

The problem is you do not check the result of fgets() for a failure (i.e. end of file).

// if there is no data left to read.
// then fgets() will return NULL and this is
// equivalent to false for a while loop and thus it will
// not enter the loop.
while (fgets(row, MAX_LINE, file)) {

    // Have successful read a row.
    );
    
    // Parse row
    ....

    // Use the variables
}

Summary: Always check that your read worked.

Tokenize string without including the NULL terminator?

1 Answers1