1

A lot of people here have asked how to convert unsigned/signed integer/long to C string.

And the most common answer is use sprintf (or snprintf). However, it requires different format strings for different types (e.g. uint32_t, int32_t, uint64_t, int64_t etc).

And I have a function template like this:

// T can only be uint16_t, int16_t, uint32_t, int32_t, uint64_t, int64_t
template <type T>
void foo(char* buffer, T value) {
// write value to buffer
}

I can specialize the function template to solve my problem.

I just wonder whether there is a more elegant and efficient way (i.e. without a temporary buffer like stringstream).

Thanks!

too honest for this site
  • 12,050
  • 4
  • 30
  • 52
Hei
  • 1,844
  • 3
  • 21
  • 35
  • Neither. Just happened that I run into it at work and curious whether there is a brilliant solution :) – Hei Apr 26 '17 at 04:32
  • There are undoubtedly brilliant solutions. Have you determined that the existing solutions you've seen are not fast enough? The solutions available in the STL are undoubtedly very well thought out and work under all conditions. As is pointed out in one of the answers, even the function signature you provide is fragile - there's no guarantee that the buffer you provide the function is big enough – Arunas Apr 26 '17 at 05:12
  • The cost of such an operation is complicated. Are you hoping for minimal time? Minimal memory? minimal code? do you have some expected distribution of integers, e.g. most will be under 65535, do you have incredible amount of memory available? – Arunas Apr 26 '17 at 05:46
  • @Hei check out my hand optimized [to_str](http://stackoverflow.com/a/43625108/468725), I'm curious how much faster that would be than your second best version. – Pavel P Apr 26 '17 at 15:21

2 Answers2

3

Perhaps the simplest implementation prior to C++17 would be:

std::strcpy(buffer, std::to_string(value).c_str());

This does require a temporary buffer (a temporary std::string) but I would be hesitant to prematurely optimize. In my opinion, this would be the most elegant way to do the conversion -- it's simple and easily understandable.

(Note that it's not possible with your function signature to ensure that the buffer pointer points to an allocation large enough to hold the stringified value.)

In C++17, you can just use std::to_chars() (you will have to add the null terminating character yourself with this function; the documentation does not state that it adds one for you).


Perhaps there is middle ground where you can declare a trait to obtain the printf-style format specifier for each numeric type?

#include <cstdio>

template <typename T>
struct printf_specifier
{
    static char const * const value;
};

template<> char const * const printf_specifier<char>::value = "%hhd";
template<> char const * const printf_specifier<unsigned char>::value = "%hhu";
template<> char const * const printf_specifier<short>::value = "%hd";
template<> char const * const printf_specifier<unsigned short>::value = "%hu";
template<> char const * const printf_specifier<int>::value = "%d";
template<> char const * const printf_specifier<unsigned int>::value = "%u";
template<> char const * const printf_specifier<long>::value = "%ld";
template<> char const * const printf_specifier<unsigned long>::value = "%lu";
template<> char const * const printf_specifier<long long>::value = "%lld";
template<> char const * const printf_specifier<unsigned long long>::value = "%llu";
template<> char const * const printf_specifier<float>::value = "%f";
template<> char const * const printf_specifier<double>::value = "%f";

template <typename T>
void foo(char *buffer, T value)
{
    std::sprintf(buffer, printf_specifier<T>::value, value);
}

I would, however, suggest using snprintf since it won't overrun your buffer if you give it the number of characters it is allowed to write:

template <typename T>
int foo(char *buffer, std::size_t size, T value)
{
    return std::snprintf(buffer, size, printf_specifier<T>::value, value);
}

If even this is too much bloat, you can just do the conversion entirely yourself:

#include <algorithm>
#include <cstdlib>

template <typename T>
void foo(char *buffer, T value)
{
    static_assert(std::is_integral<T>::value, "Type of value must be an integral type");

    if (value < 0) {
        *(buffer++) = '-';
    }

    char *start = buffer;

    while (value != 0) {
        *(buffer++) = '0' + std::abs(value % 10);
        value /= 10;
    }

    if (buffer == start) {
        *(buffer++) = '0';
    } else {
        std::reverse(start, buffer);
    }

    *buffer = '\0';
}

It could be faster to use log10 to figure out how long the string will be and write it from back to front instead of writing it backwards and then reversing it, but I'll leave that option as an exercise for you if you deem it necessary.

cdhowie
  • 158,093
  • 24
  • 286
  • 300
  • Thanks for your quick reply. Agree on avoiding all premature optimizations. But it is on my critical path of my application (requiring to be very fast...< 2 microseconds...so everything counts). So I would like to see how others will implement :) – Hei Apr 26 '17 at 04:17
  • Unfortunately, I am limited to GCC 4.8.x, which doesn't even implement the complete C++11 (although I put a C++11 tag here). – Hei Apr 26 '17 at 04:18
  • @Hei I added an approach that might work well for you. – cdhowie Apr 26 '17 at 04:31
  • @cdhowie that's by the way how MS std::to_string is implemented (AFAIK) – Pavel P Apr 26 '17 at 04:54
  • 1
    @Hei I'd recommend avoiding snprintf/printf if you're worried about your critical path. It loads a lot of code into the instruction cache, which can affect latency. If your application is sensitive to losing 100s of nanoseconds, you may want to measure and compare rolling your own. Especially if you only need to convert integers. – Shalom Craimer Apr 26 '17 at 05:29
  • @scraimer then just use `itoa`. IMO sprintf isn't that bad, std::to_string uses it. – Pavel P Apr 26 '17 at 05:59
  • @Pavel I'm scarred from that one time I decided to use snprintf and my embedded application swelled by over 35K. So... Don't trust me! Measure, measure, measure! (Latency or whatever is important to you.) – Shalom Craimer Apr 26 '17 at 10:43
  • @scraimer well, if 35K was important to OP I guess he would clearly mentioned here and all kinds of std::to_string would most certainly be out of question from the start. On top of that with `GCC 4.8.x` dynamic linking could be used with no size issues. – Pavel P Apr 26 '17 at 14:02
  • @Hei See my additional approach, just doing the conversion yourself without any library assistance (aside from `std::reverse`). – cdhowie Apr 26 '17 at 14:28
  • @cdhowie thanks for your input! really appreciate it. Your last solution has a bug that I ran into (that indirectly made me start his post) -- "value = -value" will fail when value is std::numeric_limits::min() (assuming that the machine is still 2's complement). – Hei Apr 26 '17 at 14:56
  • @Hei Good catch. Fixed. – cdhowie Apr 26 '17 at 15:14
  • @cdhowie try to compare performance of your foo with my [to_str](http://stackoverflow.com/a/43625108/468725). My quick measurements over 50% speed up improvement. – Pavel P Apr 26 '17 at 17:03
  • @cdhowie your code is very readable, clean and elegant. Just that it is a bit slow by comparing to Pavel's solution. Thanks for your solution! – Hei Apr 27 '17 at 08:58
1

Just use sprintf or itoa (non portable):

char* append_num(char* buf, int n)
{
    return buf + sprintf(buf, "%d", n);
}

some std::to_string implementations actually use sprintf and copy result to a new std::string.

Here's something that could be considered well optimized. The difference from regular itoa is that it does twice less integer divisions which aren't trivial instructions on most CPUs.

static int log10_1(unsigned int num)
{
    int ret;
    static_assert(sizeof(num) == 4, "expected 32-bit unsigned int");
    // extend this logic for 64 bit numbers
    if (num >= 10000)
    {
        if (num >= 1000000)
        {
            if (num >= 100000000)
                ret = (num >= 1000000000) ? 10 : 9;
            else
                ret = (num >= 10000000) ? 8 : 7;
        }
        else
            ret = (num >= 100000) ? 6 : 5;
    }
    else
    {
        if (num >= 100)
            ret = num >= 1000 ? 4 : 3;
        else
            ret = num >= 10 ? 2 : 1;
    }
    return ret;
}

// write string representation of number `n` into buf and return pointer to rterminating null
char* to_str(char* buf, unsigned int n)
{
    static const char dig_[] = "0001020304050607080910111213141516171819"
        "20212223242526272829303132333435363738394041424344454647484950515253545556575859"
        "60616263646566676869707172737475767778798081828384858687888990919293949596979899";
    int len = log10_1(n);
    char *p = buf + len;
    *p-- = 0;
    while (n >= 100)
    {
        unsigned int x = (n % 100) * 2;
        n /= 100;
        *p-- = dig_[x + 1];
        *p-- = dig_[x];
    }
    if (n >= 10)
    {
        unsigned int x = n * 2;
        *p-- = dig_[x + 1];
        *p-- = dig_[x];
    }
    else
        *p-- = (char)('0' + n);
    return buf + len;
}

// write string representation of number `n` into buf and return pointer to terminating null
char* to_str(char* buf, int n)
{
    unsigned int l;
    if (n < 0)
    {
        *buf++ = '-';
        if (n == INT_MIN)
        {
            static_assert(sizeof(n) == 4, "expected 32-bit int");
            memcpy(buf, "2147483648", 10);
            return buf + 10;
        }
        l = (unsigned int)(-n);
    }
    else
        l = (unsigned int)n;
    return to_str(buf, l);
}

to_str is more than twice as fast compared to cdhowie's foo and about 6 times as fast as sprintf. Compare times:

foo time: 745 ms
to_str time: 327 ms
sprintf time: 1989 ms

There is already a good stackoverflow page for optimized to_string function: C++ performance challenge: integer to std::string conversion. Fastest algorithm is essentially identical to mine.

Community
  • 1
  • 1
Pavel P
  • 15,789
  • 11
  • 79
  • 128
  • if you knew the length of the string before creating it for example with something from here http://stackoverflow.com/questions/9721042/count-number-of-digits-which-method-is-most-efficient , you could avoid the `memcpy`. As well, one could extend this one more step to look at groups of 3 digits, 4, etc - might be a marginal speed improvement, but increase memory consumption, albeit only once. – Arunas Apr 26 '17 at 15:59
  • @Arunas yes, the memcpy step could be improved easily to copy using multiple bytes etc. Still even with the memcpy this function performs pretty well. – Pavel P Apr 26 '17 at 16:02
  • you're on a roll! Now, of course there are two divisions happening, and those are bad, I wonder whether https://linux.die.net/man/3/div provides anything that would reduce the number of cycles – Arunas Apr 26 '17 at 21:30
  • though according to http://en.cppreference.com/w/cpp/numeric/math/div, 'On many platforms, a single CPU instruction obtains both the quotient and the remainder, and this function may leverage that, although compilers are generally able to merge nearby / and % where suitable. ', so there might not be much advantage. – Arunas Apr 26 '17 at 21:32
  • @Arunas Compiler will definitely merge them into one. As division is in general heavy weight optimizer will know what to do with / and %, and if div is implemented as an extern that returns values then it might harm performance. – Pavel P Apr 26 '17 at 21:54
  • @Pavel thanks for your input. I think your code has the same issue as cdhowie's previous code -- (unsigned int)(-n) is an undefined behavior if n is std::numeric_limits::min() per C11 standard. – Hei Apr 27 '17 at 05:31
  • @Hei I added step to handle `INT_MIN`, in practice though code would still return correct result even without manual check int min (on x86, arm etc). – Pavel P Apr 27 '17 at 05:58
  • Right, your previous version would break because your previous version had long as the type of n, std::numeric_limits::min() * -1 won't fit in uint32_t (unsigned int). I tried out both your code and cdhowie's code with a larger sample size (65535, I know it is still too small for benchmark). Without a proper setup (e.g. using cpuset, serialized timer, etc), cdhowie's code runs faster. – Hei Apr 27 '17 at 07:08
  • @Hei well, I [posted online sample code](https://ideone.com/7XWsWb) and without any special tuning it's more than twice faster. On MS visuals studio I also get similar results compared to `foo`, but over 10 times faster than sprintf. Can you post your benchmark code on ideone.com so I can see, otherwise it's hard to imagine how it could possibly be. Do you compile with -02? – Pavel P Apr 27 '17 at 07:18
  • @Pavel you are indeed right that your code is faster...I made a mistake when I looked at the digits on my screen. Anyway, I will get a proper benchmark environment setup first and do a proper benchmark. Thanks! – Hei Apr 27 '17 at 07:57