Tokenizing multiple String in C with KEIL Compiler

Question

I am writing on a Microcontroller-program using the Keil-Compiler. The program creates several CSV-Like Strings (Logging-Lines). For example "A;001;ERROR;C05;...\n" To save space I now want to reduce the data by just logging the differences. Therefore I am saving the last logged line and compare it to the new one. If a value in a column is the same, I want to just omit it. For example: "A;001;ERROR;C05;...\n" <- previous Log

"A;002;ERROR;C06;...\n" <- new Log

would result in ";002;;C06;...\n"

At first I just included <string.h> and used 'strtok' to step through my CSV-line. Since I need to compare 2 Strings/Lines, I would need to use it simultaneously on 2 different Strings which does not work. So I switched to 'strtok_r' which seems to not work at all:

token1 = strtok_r(m_cActLogLine, ";", pointer1);
while (token1 != NULL) {
  token1 = strtok_r(NULL, ";", pointer1);
}

This just gives me strange behaviour. Typically the second call to 'strtok_r' just returns a NULL and the loop is left. So is there maybe another way of achieving the desired behaviour?

EDIT:

To clarify what I mean, this is what I am currently trying to get to work: My Input (m_cMeasureLogLine) is "M;0001;001;01;40;1000.00;0.00;1000.00;0.00;360.00;0.00;400.00;24.90;400.00;-9999.00;-9999.00;-9999.00;0;LED;;;;;400.00;34.40;25.41;27.88;29.01;0.00;0.00;0.00;-100.00;0.00;-1000.00;-1000.00;-103.032;-70.192;19;8192.00;0.00;0;"

char m_cActLogLine[MAX_SIZE_PARAM_LINE_TEXT];
char* token1;
char* token2;
char** pointer1;
char** pointer2;
void vLogProtocolMeasureData()
{
  strcpy(m_cActLogLine, m_cMeasureLogLine);
  token1 = strtok_r(m_cActLogLine, ";", pointer1);
  while (token1 != NULL) {
    token1 = strtok_r(NULL, ";", pointer1);
  }
}

The function is part of a bigger embedded project so I dont have Console Output but use the debugger to check the contents of my variables. In the above example, after the first call to 'strtok_r' token1 is 'M' which is correct. After the second call however (in the Loop) token 1 becomes 0x00000000 (NULL).

If I instead use 'strtok' instead:

strcpy(m_cActLogLine, m_cMeasureLogLine);
token1 = strtok(m_cActLogLine, ";");
while (token1 != NULL) {
  token1 = strtok(NULL, ";");
}

the loop iterates just fine. But that way I cant process two Strings at a time and compare values column-wise.

In string.h the functions are declared as:

extern _ARMABI char *strtok(char * __restrict /*s1*/, const char * __restrict /*s2*/) __attribute__((__nonnull__(2)));
extern _ARMABI char *_strtok_r(char * /*s1*/, const char * /*s2*/, char ** /*ptr*/) __attribute__((__nonnull__(2,3)));
#ifndef __STRICT_ANSI__
extern _ARMABI char *strtok_r(char * /*s1*/, const char * /*s2*/, char ** /*ptr*/) __attribute__((__nonnull__(2,3)));
#endif

We'll need more code before we can advise you, preferably a [mcve], and *certainly* the declarations and any applicable initializations of the variables involved. — John Bollinger, Feb 13 '17 at 16:33
See http://stackoverflow.com/questions/15961253/c-correct-usage-of-strtok-r and compare to what you're doing — Russ Schultz, Feb 13 '17 at 23:26
(my guess is you're not passing a char ** as the last parameter of strtok_r()) — Russ Schultz, Feb 14 '17 at 00:48
I updated my initial question with a minimal example of what I am trying to do. The third parameter is a char** — kain, Feb 14 '17 at 10:33

score 2 · Answer 1 · answered Feb 14 '17 at 14:37

You need to pass a pointer to a valid char* for the last parameter of strtok_r(). You're passing a pointer to a pointer with pointer1, but it's a NULL (because it's a globally scoped variable that isn't assigned a value), so when strtok_r() goes to store it's iterating pointer at the address to a pointer you pass in, it's trying to write something to address 0x00000000.

Try...

char m_cActLogLine[MAX_SIZE_PARAM_LINE_TEXT];
char* token1;
char* pointer1;

void vLogProtocolMeasureData()
{
  strcpy(m_cActLogLine, m_cMeasureLogLine);
  token1 = strtok_r(m_cActLogLine, ";", &pointer1);
  while (token1 != NULL) {
    token1 = strtok_r(NULL, ";", &pointer1);
}

Tokenizing multiple String in C with KEIL Compiler

1 Answers1