istringstream not to tokenize a text between quotes

Question

I'm using istringstream to break the tokens of a string. Let's say I have a string like this:

print "this is a test"

It breaks it into 5 tokens:

print | "this | is | a | test"

where it should generate 2 tokens:

print | "this is a test"

Is there a good way to fix it? I tried doing it manually, by finding the quotes in the text and doing a substring, but it's too much work.

I guess there should be a simpler way, or is istringstream the way to do it?

http://stackoverflow.com/questions/5534620/mimicking-the-shell-argument-parser-in-c — jxh, Mar 12 '13 at 16:43

score 1 · Answer 1 · answered Mar 12 '13 at 15:01

1

Well iostreams tokenize by .. token. Tokens are separated by spaces in this instance. You're going to have to read line-by-line (or perhaps the whole thing!) and parse it properly yourself.

answered Mar 12 '13 at 15:01

Lightness Races in Orbit

378,754
76
643
1,055

score 1 · Answer 2 · answered Mar 12 '13 at 15:14

You could use wordexp to do this, but it has restrictions since it is using the command shell to do the parsing. In particular, it will react badly to shell symbols that deal with redirection. It will also do environment variable expansion, which may or may not be what you want.

wordexp_t result;
wordexp(cmdstr.c_str(), &result, 0);
for (int i = 0; result.we_wordv[i]; ++i) {
    //...
}
wordfree(&result);

Details can be found in the wordexp man page.

istringstream not to tokenize a text between quotes

2 Answers2