2

I am trying to figure out how to reuse a stringstream object without the need to re-allocate the underlying string every time I put something in the stream. I have found this answer which led me to do this:

int main() {

        stringstream ss;
        int x;
        ss << "423";
        ss >> x; // x is now 423

        ss.clear();
        ss.seekg(0);
        ss.seekp(0);

        ss << "1";
        ss >> x; // x is now 123. Instead I want x to be 1.

        std::cout << x << std::endl;
}

Unfortunately, this doesn't work since the contents of the string from the first pass are still there (after the first pass the string is "423" and after the second pass they are "123"). However, if I add a space right after the second put, things seem to work, like this:

int main() {

        stringstream ss;
        int x;
        ss << "423";
        ss >> x; // x is now 423

        ss.clear();
        ss.seekg(0);
        ss.seekp(0);

        ss << "1";
        ss << " "; // add a space right after the desired value
        ss >> x; // x is now 1

        std::cout << x << std::endl;
}

After the second pass the string is "1 3". I am not very familiar with the I/O library and I would like to know if the above approach is safe, or if it just happens to work in this trivial example, or if there are better solutions out there. Live code here. Thank you!

linuxfever
  • 3,763
  • 2
  • 19
  • 43
  • 4
    Possible duplicate of [How do you clear a stringstream variable?](https://stackoverflow.com/questions/20731/how-do-you-clear-a-stringstream-variable) – kiran Biradar Aug 24 '18 at 15:45
  • 2
    @kiranBiradar Does that stop the stringstream from allocating more though? Looks to me like it wouldn't since it is set to a default constructed string. – NathanOliver Aug 24 '18 at 15:48
  • 1
    What do you mean by allocating more memory? It will just clear the content of stringstream object. – kiran Biradar Aug 24 '18 at 15:50
  • 1
    @kiranBiradar `std::stringstream` is built on top of a `std::string`. When you add more stuff into the stream it grows the underlying `std::string`. The op wants to keep that grown string so no more memory allocation have to happen but they want the stream in an empty state. – NathanOliver Aug 24 '18 at 15:52
  • Also there is a problem of resetting numerous formatting flags/options. – HolyBlackCat Aug 24 '18 at 15:55
  • 1
    Why this strange requirement? I would create new instance of `stringstream` and do not worry about this micro-optimization. If you have performance issue first use profiler to find source of problems. – Marek R Aug 24 '18 at 15:56
  • 2
    @MarekR If you are calling this a lot it's not so strange to want to be able to reuse memory you have already acquired. – NathanOliver Aug 24 '18 at 16:01
  • 1
    @MarekR basically I am calling this millions of times among many threads and every time re-allocation takes place the heap memory gets locked and the whole thing runs as if a single-threaded application. Even when using the Hoard library I get the same behaviour. – linuxfever Aug 24 '18 at 16:03
  • in such case consider use of custom `allocator` as argument of `basic_stringstream`. I'm surprised there is no constructor of `basic_stringstream` to pass object representing specific instance of allocator. Another approach would be find some open source alternative. – Marek R Aug 24 '18 at 16:19
  • anyway are you sure that `basic_stringstream` allocates so often? IMO it should reserve larger piece of memory and keep it there, and only when `str` is called allocate new memory or when buffer should be bigger. – Marek R Aug 24 '18 at 16:23
  • @MarekR yes, I have tested it. Yes, it should reserve larger piece and keep it there (similarly to what a vector does), but it doesn't seem to be the case. Anyway, what do you think of the approach in the question (adding an extra space)? Will that work or is it unsafe? Thanks – linuxfever Aug 24 '18 at 16:29
  • @linuxfever Your space solution should work. You could put a newline in the buffer if you ever want to use getline for some reason. – NathanOliver Aug 24 '18 at 16:32
  • 1
    @MarekR minimizing string allocations in hot loops is not micro-optimization, it's generally what makes a difference between code that crawls and code that runs. Keeping an `std::string` out of the loop and doing `clear()` on it instead of re-constructing it every time is performance 101 when parsing stuff. You go from one allocation per iteration (O(n)) to the expected amortized O(1) behavior. – Matteo Italia Aug 25 '18 at 08:48

1 Answers1

3

I did some investigations and experiments using clang with this code:

Test code

class LogHelper {
public:
    ~LogHelper() {
        std::cout << out.str() << '\n';
    }

    std::ostream &stream() {
        return out;
    }

private:
    std::ostringstream out;
};

#define LOG() LogHelper().stream() << __FUNCTION__ << '(' << __LINE__ << ")"
#define VAR(x) ", " #x "[" << x << ']'

class MyAllocator : public std::allocator<char> {
public:
    using base = allocator<value_type>;
    using base::allocator;

    value_type* allocate( std::size_t n, const void * hint) {
        LOG() << VAR(n);
        return base::allocate(n, hint);
    }

    value_type* allocate( std::size_t n ) {
        LOG() << VAR(n);
        return base::allocate(n);
    }

    void deallocate( value_type* p, std::size_t n ) {
        LOG() << VAR(n);
        base::deallocate(p, n);
    }
};

using MySStream = std::basic_stringstream<char, std::char_traits<char>, MyAllocator>;
using MyString = std::basic_string<char, std::char_traits<char>, MyAllocator>;

int main() {
    MySStream ss; // (MyString(255, '\0'));
    ss.clear();

    int x;
    ss << "423";
    ss << " 423";

    LOG();
    ss << " 423jlfskdfjl jfsd sdfdsfkdf dsfg dsfg dfg dfg dsfg df gdf gdfg dsfg dsfgdsfgds";
    LOG();
    ss >> x;

    ss.clear();
    ss.str({});

    ss.seekg(0);
    ss.seekp(0);

    ss << "1";
    ss >> x;

    std::cout << x << std::endl;
    LOG();

    return 0;
}
main(55)
allocate(34), n[48]
allocate(34), n[96]
deallocate(39), n[48]
main(57)
1
main(70)
deallocate(39), n[96]
allocate(34), n[256]
allocate(34), n[256]
deallocate(39), n[256]
main(55)
main(57)
1
main(70)
deallocate(39), n[256]

And I have couple findings

  1. clang and visual behave in same meaner, gcc has some stang issues with this code.
  2. std::basic_stringstream string buffers always grows, never shrinks
  3. std::basic_stringstream sucks. You can't reserve string size or buffer size, like in case std::string. Custom allocator can be passed only by type, you can't provide allocator by object.
  4. To reduce allocations you have to set large string at the begging, then until you you will not succeed its capacity re-allocation will not happen (second example).
  5. providing a custom allocator doesn't help a lot and it adds boiler plate code when fetching result string. In my examples it is used mainly to log allocations and deallocations.
  6. ss.str({}); do not cause allocation. Here small string optimization helps

Conclusions:

  1. you can safely do ss.str({}); as recommended in linked by you SO answer and it will not cause allocation. Here small string optimization helps and the fact that
  2. custom allocator is not very helpful
  3. setting large dummy string at begging is quite effective evil hack
  4. finding alternative should be better approach (maybe boost - I didn't test it)
  5. Point 1 and your question shows that you didn't do any measurements and your question is based on personal assumptions.
Marek R
  • 32,568
  • 6
  • 55
  • 140
  • This is great, thank you. To be fair, I did test using a new stringstream object every time and noticed that reallocations happen all the time. However, I did assume that `ss.str({})` will completely clear the underlying string based on https://en.cppreference.com/w/cpp/io/basic_stringstream/str which states that the contents of the underlying string are replaced with that of the argument (which to me that implies reallocations). Thanks again – linuxfever Aug 25 '18 at 07:03
  • 1
    @linuxfever actually I'd have trusted `str("")` to do the equivalent of a `s = ""` over the underlying `std::string` - which doesn't reallocate `s` (keeping the `capacity` is kind of the point of assigning to an existing string vs creating a new one). Still, I don't know if this is specified in some way, I'd have to check the standard. – Matteo Italia Aug 25 '18 at 08:56
  • can someone explain votes down? What is wrong with my answer? If I'm wrong somewhere I wish to know where and why. – Marek R Aug 25 '18 at 09:28
  • @MarekR Your answer included exactly information that I was looking. But it was in a bit confusing format which required careful reading. It might help if there was short summary first explaining that ss.str({}) clears the stream and doesn't cause deallocations. – Pauli Nieminen Sep 20 '19 at 20:32
  • If you click on the "visual studio" link then it shows that `ss.str({})` very clearly does cause deallocations. My reading of the implementation of this in the VCRT confirms this. – Miral Jan 06 '20 at 03:59