Best way to remove white spaces from std::string

Question

I have loaded a text file content to a std::string. And I want to remove the whitespaces from the loaded string.

1. Which below method need to be used for better performance?
2. Which below method can be a best practice.?
3. Else any better way is there to achieve this?

Method 1:
Scan the string using string.find() in a loop statement and remove whitespace using string.erase();

Method 2:
Scan the string using string.find() in a loop statement and copy the non whitespace characters to a new string() variable using string.append();

Method 3:
Scan the string using string.find() in a loop statement and copy the non whitespace characters to a new string(size_t n, char c) variable using string.replace();

Method 4:
Allocate a char* (using malloc[size of the source string])
Scan the string using string.find() in a loop statement and copy the non whitespace characters to the char* variable using strcpy and then strcat();
finally copy the char* to a new string
free char*

Answer 1

Edit: upgraded to locale-aware trait. Thanks user657267 !

Standards algorithms are neat !

s.erase(std::remove_if(
    begin(s), end(s),
    [l = std::locale{}](auto ch) { return std::isspace(ch, l); }
), end(s));

Live on Coliru

That was for in-place modification, if you need to keep the original string however :

std::string s2;
s2.reserve(s.size());
std::remove_copy_if(
    begin(s), end(s),
    std::back_inserter(s2),
    [l = std::locale{}](auto ch) { return std::isspace(ch, l); }
);

Live on Coliru

Answer 2

For readability I would go with the boost string algorithm library

#include <boost/algorithm/string.hpp>
std::string str1="Hello  Dolly,   Hello World!"
boost::erase_all(str1, " ");

EDIT: Example for all whitespace characters:

std::string str1="Hello  Dolly,\n   Hello\t World!\t";
boost::find_format_all(str1, boost::token_finder(::isspace), boost::const_formatter(""));

EDIT2: I ran some benchmarks to see how this method compares with Quentin's answer. I ran 100 samples using both the locale aware and non locale aware version of isspace. The average times in micro seconds were:

| method                            | avg. time (μs) |
|-----------------------------------|----------------|
| boost::find_format_all w/locale   | 2.02429        |
| boost::find_format_all w/o locale | 0.578105588    |
| std::remove_if w/locale           | 1.197737742    |
| std::remove_if w/o locale         | 0.190661227    |

Answer 3

IMHO, best performance you can get with method 2, but before appending you need to call std::string::reserve method to set capacity of new string to the size of initial string. This needed to prevent unnecessary reallocations, when appending.

Answer 4

Method 5: Cover all edge cases (including the current locale) with the library, and make adjustments if and only if profiling shows it to be an issue.

#include <algorithm>
#include <functional>
#include <iostream>
#include <locale>
#include <string>

template<typename CharT, typename Traits, typename Allocator>
std::basic_string<CharT, Traits, Allocator>
strip_whitespace(std::basic_string<CharT, Traits, Allocator> str)
{
  str.erase(
    remove_if(str.begin(), str.end(), bind(
      std::isspace<CharT>, std::placeholders::_1, std::locale{}
    )),
    str.end()
  );
  return str;
}

int main()
{
  std::string str{"A string with \n whitespace"};
  std::cout << str << '\n' << strip_whitespace(str) << '\n';
}

Answer 5

It is not clear what exactly do you mean by "remove white spaces" - does it intend to make text unreadable and break source code, or do you mean "any excess whitespace"?

Since there was one answer that suggested a third party library, I will go in with a method from Qt, which will trim out only "excess whitespace":

QString s("Test\n new line,   multiple spacebars \t tab!");

qDebug() << s;
qDebug() << s.simplified();

Output:

"Test
 new line,   multiple spacebars      tab!"

"Test new line, multiple spacebars tab!"

Most of the other answers focus on removing all spacebars, but strictly speaking, there are other characters which classify as whitespace such as tab or new line.

Looking at the code for the simplified() method, I'd say it is fairly efficient:

QString QString::simplified() const
{
    if (d->size == 0)
        return *this;
    QString result(d->size, Qt::Uninitialized);
    const QChar *from = (const QChar*) d->data;
    const QChar *fromend = (const QChar*) from+d->size;
    int outc=0;
    QChar *to   = (QChar*) result.d->data;
    for (;;) {
        while (from!=fromend && from->isSpace())
            from++;
        while (from!=fromend && !from->isSpace())
            to[outc++] = *from++;
        if (from!=fromend)
            to[outc++] = QLatin1Char(' ');
        else
            break;
    }
    if (outc > 0 && to[outc-1] == QLatin1Char(' '))
        outc--;
    result.truncate(outc);
    return result;
}

The new string is preallocated without initialization, so both any initialization and any reallocations are avoided, then similarly to method 2 (keep in mind appending without reserving space will result in many and slow reallocations), only the useful data is copied to the result string, and in the end, it is trimmed shorter to remove any wasted memory.

You could follow this logic to implement it efficiently for std::string

Taken directly from the Qt source code.

Answer 6

For performance it would be better to read the string one platform word at a time (8 bytes on 64-bit platforms), then extract each character from the register read, test it for whitespace and if it is not a whitespace, add it to the platform word wide register to be written next. When the register to be written is full (as many characters are stored as can fit into the register), write it to the output buffer allocated in advance. Character-by-character scan is 8 times slower for single-byte character strings.

Finally, the tail of the string may contain less characters than would occupy a full platform word. A counter is needed then to know how many characters are in the last platform word processed.