2

What is most efficient way to read a file of integer in each line without opening it? I have a file with ONLY integer in each line, ie: num.txt

100
231 
312
...

In my program, I use while loop to read it;

int input = 0;
while(cin >> input)
{        
// Assignment
}

I use time a.out <num.txt to read it in Linux It turns out that it will take about 15 second (user time) to read 100 million numbers. So I was wondering is there any better way to decrease user time?

Thank you in advance!

qdii
  • 12,505
  • 10
  • 59
  • 116
idontknoooo
  • 261
  • 1
  • 3
  • 12
  • 3
    Don't use iostreams if you care about performance. – tux3 Mar 07 '15 at 16:48
  • Here you go: http://stackoverflow.com/questions/9371238/why-is-reading-lines-from-stdin-much-slower-in-c-than-python?rq=1 – indiv Mar 07 '15 at 16:49
  • Just so you're not misleading yourself technically, you're actually using the shell to open the file when you use the redirect '<', and attaching the file to file descriptor 0, or `stdin`. If you want low-level control over exactly how the OS interacts with the file, use the bare OS calls `open(2)` and `read(2)`. – BaseZen Mar 07 '15 at 16:50
  • 1
    @tux3: We have, several times, shown that iostreams are pretty much identical to other methods of reading files. The only time there is a big difference is if you can map the file directly into memory. (Which most like won't work if you are feeding it through stdin) - you may have to turn of `sync_with_stdio` to achieve reasonable performance. – Mats Petersson Mar 07 '15 at 16:50
  • @MatsPetersson I'm sure it's easy to find plenty of other slow methods of reading files. What I'm trying to say is that by default iostreams doesn't have great performance. – tux3 Mar 07 '15 at 16:53
  • @tux3 what do you suggest as an alternative if you care about performance? – qdii Mar 07 '15 at 16:54
  • @Tux3: No, if you disable `sync_with_stdio` - at least on Linux and turn on optimisation in the compiler, `iostream` is as fast as reading large lumps of time with `fread` or `read` and parsing the integers with `strtol` or similar. – Mats Petersson Mar 07 '15 at 16:55
  • 1
    @qdii That answer by shauryachats is great imo, it's simple and reasonably fast. – tux3 Mar 07 '15 at 16:55
  • @MatsPetersson I agree. Perhaps I didn't express myself very well. – tux3 Mar 07 '15 at 16:55
  • Here's one discussion on speed of `fstream` (which for the purposes of this discussion is identical to `iostream`): http://stackoverflow.com/questions/26095160/why-are-stdfstreams-so-slow/26095604#26095604 and another: http://stackoverflow.com/questions/15115943/what-is-the-best-efficient-way-to-read-millions-of-integers-separated-by-lines-f – Mats Petersson Mar 07 '15 at 17:05
  • I'm calling this a duplicate of the second one I just posted above. – Mats Petersson Mar 07 '15 at 17:09

1 Answers1

5
int input = 0;
ios_base::sync_with_stdio(false); 
//Add this statement and see the magic!
while(cin >> input)
{        
// Assignment
}

To make it ultra fast (Not recommended for assignments!), use getchar_unlocked():

int read_int() {
  char c = getchar_unlocked();
  while(c<'0' || c>'9') c = gc();
  int ret = 0;
  while(c>='0' && c<='9') {
    ret = 10 * ret + c - 48;
    c = getchar_unlocked();
  }
  return ret;
}

int input = 0;
while((input = read_int()) != EOF)
{        
// Assignment
}

Vaughn Cato's answer explains it beautifully.

Community
  • 1
  • 1
shauryachats
  • 9,975
  • 4
  • 35
  • 48