Pointer moving inside string

Question

I'm strudying this code but I'm not understanding how the pointer is moving inside of buffer

...

  while(fgets(buffer,buf_size,fp) != NULL){  
    read_line_p = malloc((strlen(buffer)+1)*sizeof(char));   
    strcpy(read_line_p,buffer);   
    char *string_field_in_read_line_p = strtok(read_line_p,",");
    char *integer_field_in_read_line_p = strtok(NULL,",");  

    char *string_field_1 = malloc((strlen(string_field_in_read_line_p)+1)*sizeof(char));
    char *string_field_2 = malloc((strlen(string_field_in_read_line_p)+1)*sizeof(char));  

    strcpy(string_field_1,string_field_in_read_line_p);
    strcpy(string_field_2,string_field_in_read_line_p);    
    int integer_field = atoi(integer_field_in_read_line_p);  

    struct record *record_p = malloc(sizeof(struct record));   
    record_p->string_field = string_field_1;
    record_p->integer_field = integer_field;

    ordered_array_add(array, (void*)record_p);

    free(read_line_p);
  }

...

The source code does this :

reads from a .csv file millions of records composed by a string and an integer which are separated by , and every record is placed on a different line; every record is added as a singol element into a generic array that we must order. The generic array is rappresented by

typedef struct {
  void** array; 
  unsigned long el_num; //index
  unsigned long array_capacity; //length
  int (*precedes)(void*,void*); //precedence relation (name of a function in main which denota which one field we're comparing)
}OrderedArray;

Inside of this struct we have the precedes function which tells us if we have to sort the array by the string field or by the integer field.

Example of records inside our csv file

firstword , 10

secondword , 9

thirdword, 8 ecc..

So at each execution of ordered_array_add we will have inserted a new element in the array.

Follows ordered_array_add

void ordered_array_add(OrderedArray *ordered_array, void* element){
  if(element == NULL){
    fprintf(stderr,"add_ordered_array_element: element parameter cannot be NULL");
    exit(EXIT_FAILURE);
  }

  if(ordered_array->el_num >= ordered_array->array_capacity){
    ordered_array->array = realloc(ordered_array->array,2*(ordered_array->array_capacity)*sizeof(void*));
    if(ordered_array->array == NULL){
      fprintf(stderr,"ordered_array_add: unable to reallocate memory to host the new element");
      exit(EXIT_FAILURE);
    }
    ordered_array->array_capacity = 2*ordered_array->array_capacity;
  }

  unsigned long index = get_index_to_insert(ordered_array, element);

  insert_element(ordered_array,element,index);

  (ordered_array->el_num)++;

}

I don't understand how the first loop scannes through the string buffer because i don't see any indexes in the mentioned loop.

I wrote a similar code to the first loop that I posted, the issue is that it stops after reading the first word from buffer while the code that I'm studying reads successfully the whole string

while(fgets(buffer,buf_size,fp) != NULL) {
char *word = strtok(buffer, " ,.:");

    add(words_to_correct, word);
    words_to_correct->el_num = words_to_correct->el_num+1;
    printf("%s\n", word);

}

Answer 1

Your entire first loop can be rewritten as:

while(fgets(buffer,buf_size,fp) != NULL){  
    // note how sizeof() is used - that way if the type of
    // record_p changes, no changes to this code are needed
    struct record *record_p = malloc(sizeof(*record_p));   

    // no need at all for temporary copies of the strings
    record_p->string_field = strdup(strtok(buffer,","));
    record_p->integer_field = atoi(strtok(NULL,","));

    ordered_array_add(array, (void*)record_p);
  }

There's no need to call malloc() and strcpy() so many times - and that pair can be replaced with strdup() - that's both POSIX-standard and supported on Windows so it's pretty widely available.

Of course, that code needs error checking and it shouldn't be using atoi() at all, but as posted here it will duplicate your original functionality.

With the added benefit that you can actually tell what's going on.

Your code

while(fgets(buffer,buf_size,fp) != NULL) {
char *word = strtok(buffer, " ,.:");

    add(words_to_correct, word);
    words_to_correct->el_num = words_to_correct->el_num+1;
    printf("%s\n", word);

}

will only process the first word in each line - you need to keep calling strtok() until it returns NULL, because strtok() only returns one token:

while(fgets(buffer,buf_size,fp) != NULL) {
    // trick to keep loop simple - start by using
    // buffer on the first loop iteration, then
    // set tmp to NULL so later iterations works too
    char *tmp = buffer;
    // loop until strtok() returns null
    for ( ;; )
    {
        // note use of tmp here
        char *word = strtok(tmp, " ,.:");

        // line is fully parsed - break this loop
        // and get the next line to parse
        if (word == NULL)
        {
            break;
        }

        // now set tmp to NULL so next strtok()
        // gets a NULL first parameter
        tmp = NULL;

        add(words_to_correct, word);
        words_to_correct->el_num = words_to_correct->el_num+1;
        printf("%s\n", word);
    }

}

Note also that I'm spreading things out and not trying to stuff as much code on each line. That's usually a lot easier to read.