9

Lot of profiling shows that C++ streams are not the best way to performe file or text string manipulation when performance (speed) is required. Still, the standard streams are a good way to keep things type-safe.

From what I've read, most of the problem is because streams implementations have to 1) create/copy a lot of little objects 2) arnt fully generic (don't manage char and wchar the same way?) etc.

Anyway, I was thinking that maybe some C++0x would allow implementers to limit at least object creation/copy and maybe there are other features that would allow other performance improvements, maybe allowing to reach the printf() performance?

Is there an immediate impact? Or will we have to wait for new implementations? Or do we still need a new (STL-like) stream library?

Klaim
  • 67,274
  • 36
  • 133
  • 188
  • Just a side note about the subject. It is my opinion that IO libraries are super hard to design. I would like any one to point an IO library in any language that is 1) safe, 2) efficient, 3) portable and 4) usable by an average programmer(like me for example!). I am not saying that designing one with the above conditions is impossible, I am just saying it is harder than what most programmers think. Btw, +1 for the question. – Khaled Alshaya Dec 10 '10 at 14:04
  • I agree. I guess that's the main reason why there is Boost.IOStream existing to help implementing streams. I've heard some people say that the main problems with the standard stream is that they werent thought with template/genericity because there was no template feature at the time it was implemented. They often suggest that designing a new stream library based on genericity like STL would solve a lot of performance problems. – Klaim Dec 10 '10 at 14:11
  • IOstreams were reimplemented for the Standard. One issue I had in 2002 in converting a pre-Standard C++ system to a standards-conforming one was the change in what some of those were, when they became templated classes and functions. – David Thornley Dec 10 '10 at 17:40

1 Answers1

3

You might be interested in some of the performance comparisons in my question here. Even the lowest level functions in the C++ standard library streams API are incredibly slow under common implementations, and looking through the source code of e.g. Visual C++'s stringbuf class, I don't see copying of small temporary objects. So rvalue-references are not likely to help much.

AFAICT, the main reason for slowness of C++ iostreams is that library developers are stuck with a mindset that I/O is the bottleneck, so there's no point in worrying about performance of the I/O library. But I/O is decidedly not the bottleneck.

Community
  • 1
  • 1
Ben Voigt
  • 277,958
  • 43
  • 419
  • 720
  • Yeah. I/O is the bottleneck ... but the OS kernel handles it with buffering. Wasting time in the I/O libraries is just a waste of time. – Zan Lynx Dec 10 '10 at 17:32
  • @Zan: It came as a surprise to me too when I first benchmarked C++ iostreams, but they are so slow that even on a cold disk cache, I/O is not the bottleneck. – Ben Voigt Dec 10 '10 at 17:38
  • @Ben: That is so far against my own experiences that I don't believe it. Have you called ios::sync_with_stdio(false)? If you do input and output at the same time have you called cin.tie(0) to disable output flush? Have you made sure you are using << '\n' instead of << endl on line endings? – Zan Lynx Dec 13 '10 at 18:16
  • @Ben: Just for more information, I have C++ software using iostreams that can process many gigabytes of text in 10-20 seconds. So it cannot be that slow... – Zan Lynx Dec 13 '10 at 18:18
  • @Zan: Using what compiler and standard library implementation? My performance summary is of course only applicable to the particular toolchains I tested. Go read my linked question if you want to know about the test code, it doesn't save either of us any time for me to duplicate it here. – Ben Voigt Dec 13 '10 at 18:21