c++ stringstream is too slow, how to speed up?

Question

Possible Duplicate:
Fastest way to read numerical values from text file in C++ (double in this case)

#include <ctime>
#include <cstdlib>
#include <string>
#include <sstream>
#include <iostream>
#include <limits>

using namespace std;

static const double NAN_D = numeric_limits<double>::quiet_NaN();

void die(const char *msg, const char *info)
{
    cerr << "** error: " << msg << " \"" << info << '\"';
    exit(1);
}

double str2dou1(const string &str)
{
    if (str.empty() || str[0]=='?') return NAN_D;
    const char *c_str = str.c_str();
    char *err;
    double x = strtod(c_str, &err);
    if (*err != 0) die("unrecognized numeric data", c_str);
    return x;
}

static istringstream string_to_type_stream;

double str2dou2(const string &str)
{
    if (str.empty() || str[0]=='?') return NAN_D;
    string_to_type_stream.clear();
    string_to_type_stream.str(str);
    double x = 0.0;
    if ((string_to_type_stream >> x).fail())
        die("unrecognized numeric data", str.c_str());
    return x;
}

int main()
{
    string str("12345.6789");

    clock_t tStart, tEnd;

    cout << "strtod: ";
    tStart=clock();

    for (int i=0; i<1000000; ++i)
        double x = str2dou1(str);

    tEnd=clock();
    cout << tEnd-tStart << endl;

    cout << "sstream: ";
    tStart=clock();

    for (int i=0; i<1000000; ++i)
        double x = str2dou2(str);

    tEnd=clock();
    cout << tEnd-tStart << endl;

    return 0;
}

strtod: 405
sstream: 1389

update: remove undersocres, env: win7+vc10

Answer 1

C/C++ text to number formatting is very slow. Streams are horribly slow but even C number parsing is slow because it's quite difficult to get it correct down to the last precision bit.

In a production application where reading speed was important and where data was known to have at most three decimal digits and no scientific notation I got a vast improvement by hand-coding a floating parsing function handling only sign, integer part and any number of decimals (by "vast" I mean 10x faster compared to strtod).

If you don't need exponent and the precision of this function is enough this is the code of a parser similar to the one I wrote back then. On my PC it's now 6.8 times faster than strtod and 22.6 times faster than sstream.

double parseFloat(const std::string& input)
{
    const char *p = input.c_str();
    if (!*p || *p == '?')
        return NAN_D;
    int s = 1;
    while (*p == ' ') p++;

    if (*p == '-') {
        s = -1; p++;
    }

    double acc = 0;
    while (*p >= '0' && *p <= '9')
        acc = acc * 10 + *p++ - '0';

    if (*p == '.') {
        double k = 0.1;
        p++;
        while (*p >= '0' && *p <= '9') {
            acc += (*p++ - '0') * k;
            k *= 0.1;
        }
    }
    if (*p) die("Invalid numeric format");
    return s * acc;
}

Answer 2

string stream is slow. Quite very slow. If you are writing anything performance critical that acts on large data sets ( say loading assets after a level change during a game ) do not use string streams. I recommend using the old school c library parsing functions for performance, although I cannot say how they compare to something like boost spirit.

However, compared to c library functions, string streams are very elegant, readable and reliable so if what you are doing is not performance ciritcal I recommend sticking to streams.

Answer 3

In general, if you need speed, consider this library:

http://www.fastformat.org/

(I'm not sure if it contains functions for converting strings or streams to other types, though, so it may not answer your current example).

For the record, please note you're comparing apples to oranges here. strtod() is a simple function that has a single purpose (converting strings to double), while stringstream is a much more complex formatting mechanism, which is far from being optimized to that specific purpose. A fairer comparison would be comparing stringstream to the sprintf/sscanf line of functions, which would be slower than strtod() but still faster than stringstream. I'm not exactly sure what makes stringstream's design slower than sprintf/sscanf, but it seems like that's the case.

Answer 4

Have you considered using lexical_cast from boost?

http://www.boost.org/doc/libs/1_46_1/libs/conversion/lexical_cast.htm

Edit: btw, the clear() should be redundant.