C - fastest way to read large amounts of integers?

Question

I currently use a loop with scanf("%d", &value), but I would need it to go faster. The amount of data can be as much as 2 000 000 values. Is there any way to speed this up? I read about strtok and strtol, but I do not know how to use them and if they even would achieve the speed up I need.

Answer 1

If you want only speed and no error-checking, you can make your own function for taking an input and parsing it as an integer using getchar().

void fast_input(int* int_input)
{
    *int_input=0;
    char next_char=0;
    while( next_char < '0' || next_char > '9' ) // Skip non-digits
        next_char = getchar();
    while( next_char >= '0' && next_char <= '9' )
    {
        (*int_input) = ((*int_input)<<1) + ((*int_input)<<3) + next_char - '0';
        next_char = getchar();
    }
}

int main()
{
    int x;
    fast_input(&x);
    printf("%d\n",x);
}  

Answer 2

According to my experiences, memory mapped access is much faster for reading large amount of content from a file.

This can be achieved by

   #include <sys/mman.h>
   void *mmap(void *addr, size_t length, int prot, int flags,
              int fd, off_t offset);
   int munmap(void *addr, size_t length);

... on *Nix and some combination of

 CreateFileMapping
 OpenFileMapping
 MapViewOfFile
 MapViewOfFileEx
 UnmapViewOfFile
 FlushViewOfFile
 CloseHandle

... on Windows (refer to the link here.

Basically you want something like:

int fd = open( "filename" , 0 );
char* ptr = mmap( 0 , 4096*1024 // MAX FILE SIZE
         , PROT_WRITE | PROT_READ , MAP_PRIVATE , fd , 0 //offset
 );
// NOW READ AS IF ptr IS THE HEAD OF SOME STRING
char * thisp = ptr ;
while ( thisp != ptr+4096*1024 && *thisp ){
      int some_int_you_want = strtol( thisp , &thisp , 10 );
}
munmap(ptr,4096*1024);

I'm not very confident that the code above is correct but it should have the correct idea....