Parsing nested lists with Boost.Spirit

Question

I am trying to parse nested lists of numbers with Boost.Spirit. This is what I have so far:

//define a test string
std::string string = "[[\t1 , 2 ], [3, 4, 65.4]]";
auto it = string.begin();

//parse the string
std::vector<std::vector<double>> vector;
auto listRule = "[" >> (qi::double_ % ",") >> "]";
auto list2Rule = "[" >> (listRule % ",") >> "]";
bool match = qi::phrase_parse(it, string.end(), list2Rule, ascii::space, vector);

//check if we have a match
std::cout << "matched: " << std::boolalpha << match << '\n';
if (it != string.end())
    std::cout << "unmatched part: " << std::string{it, string.end()} << '\n';

//print the result
std::cout << "result\n";
for (const auto& v : vector) {
    std::cout << "[";
    for (double i : v)
        std::cout << i << ",";
    std::cout << "]\n";
}

The above works wonderfully and prints:

matched: true
result
[1,2,]
[3,4,65.4,]

The problem I am facing is that it does not accept empty lists. For example, by changing the string like so:

std::string string = "[[\t1 , 2 ], [3, 4, 65.4], []]";

Then I have no match (that is match == false and it == string.begin()). The vector still gets populated, apparently, but the last empty list is missing. Can anyone provide an explanation on why this is the case, and how to fix it?

The `%` parser is defined as: "List. Parse a delimited b one or more times". IIRC, you can simply mark the list as optional, and it'll do what you want : `"[" >> -(listRule % ",") >> "]";` — , Aug 23 '17 at 21:13
Uhmm.. nope, does not seem to work. In that case (and with all combinations in both `listRule` and `list2Rule` I get a list of vectors, each with only one value... — matpen, Aug 23 '17 at 21:29

sehe · Accepted Answer · 2017-08-24T08:04:29.513

You're using auto for proto expression templates in the Qi domain - that's undefined behaviour 99.9% of the time:

undefined behaviour somewhere in boost::spirit::qi::phrase_parse

Now, while you fix that, also make the list body optional:

Live On Coliru

#include <boost/spirit/include/qi.hpp>
namespace qi = boost::spirit::qi;

int main() {
    using It      = std::string::const_iterator;
    using Skipper = qi::space_type;

    for(std::string const input : { "[[\t1 , 2 ], [3, 4, 65.4]]", "[[\t1 , 2 ], [3, 4, 65.4], []]", "[]" })
    {
        std::cout << " ---- '" << input << "' ----\n";
        auto it = input.begin();

        //parse the string
        using doubles = std::vector<double>;
        using vectors = std::vector<doubles>;

        qi::rule<It, doubles(), Skipper> doubles_ = "[" >> -(qi::double_ % ",") >> "]";
        qi::rule<It, vectors(), Skipper> vectors_ = "[" >> -(doubles_    % ",") >> "]";

        vectors data;
        bool match = qi::phrase_parse(it, input.end(), vectors_, qi::space, data);

        //check if we have a match
        std::cout << "matched: " << std::boolalpha << match << '\n';
        if (it != input.end())
            std::cout << "unmatched part: " << std::string{it, input.end()} << '\n';

        //print the result
        std::cout << "result\n";
        for (const auto& v : data) {
            std::cout << "[";
            for (double i : v)
                std::cout << i << ",";
            std::cout << "]\n";
        }
    }
}

Prints

 ---- '[[   1 , 2 ], [3, 4, 65.4]]' ----
matched: true
result
[1,2,]
[3,4,65.4,]
 ---- '[[   1 , 2 ], [3, 4, 65.4], []]' ----
matched: true
result
[1,2,]
[3,4,65.4,]
[]
 ---- '[]' ----
matched: true
result

Thank you for your answer. However, this is not the output that I was expecting... see my modified question. — matpen, Aug 24 '17 at 07:52
Damn. I should pay more attention to the output next time. Obviously, that's not right :) — sehe, Aug 24 '17 at 08:03
Fixed. The problem is too much attribute compatibility magic for this case :) — sehe, Aug 24 '17 at 08:04
Yes, that works perfectly! So there where three points missing: (a) using `auto` with rules is undefined behavior, (b) a parser can be made optional with `-` and (c) each rule can have its own attribute type based on how its parser actually works. Thanks a lot! — matpen, Aug 24 '17 at 08:15

Parsing nested lists with Boost.Spirit

1 Answers1