How can I parse list with newline in boost::spirit::qi?

Question

I'm wondering how can I parse python style list with newline in boost::spirit::qi.

A basic parser will be:

list_ = '[' >> -(test_ % ',') >> -lit(",") >> ']';

With boost::spirit::ascii::space as the skipper.

However in my case, I would like to explicitly match newline myself in other grammar rules. As a result, I switch to boost::spirit::ascii::blank as the skipper. While I still want to allow newline in the definition of a list.

I eventually come to something like:

list_ = '[' >> -(test_ % (-eol >> ',' >> -eol)) >> -lit(",") >> -eol >> ']';

Basically I add eol at every possible position. This grammar works but looks horrible.

Is there a cleaner way to achieve this? Or to be more specific, if it is possible to include eol as the skipper for a single rule instead of the entire grammar.

score 2 · Accepted Answer · answered Nov 01 '17 at 01:31

Just use a separate skipper for the relevant portions.

It's a bit of a fixture from tutorials only to "bake" the skipper into the outward interface of a grammar. In 99% of the cases, that makes no sense at all: changing the skipper can wreck the grammar. So, I've taken to encapsulating the skipper the way it was intended by the gods (joke):

Instead of

template <typename It>
struct MyGrammar : qi::grammar<It, Attribute(), qi::space_type> {
    MyGrammar() : MyGrammar::base_type(start) {
        start = /*....*/;
    }
  private:
    qi::rule<It, Attribute(), qi::space_type> start;
};

I write

template <typename It>
struct MyGrammar : qi::grammar<It, Attribute()> {
    MyGrammar() : MyGrammar::base_type(start) {
        start = qi::skip(qi::space) [ mainRule ];
        mainRule = /*....*/;
    }
  private:
    qi::rule<It, Attribute()> start;
    qi::rule<It, Attribute(), qi::space_type> mainRule;
};

This means you can write

bool ok = qi::parse(f,l, MyGrammar<It>{});

and it will still use the correct skipper. Even if you did

bool ok = qi::phrase_parse(f, l, MyGrammar<It>{}, qi::char_);

it would still use the correct skipper!

Different Skippers For Different Rules

You will have to decide what will be your main skipper, but you can use the qi::skip()[] directive to switch arbitrary skippers. So let's assume that, like in most programming languages, newlines are usually insignificant whitespace, so you'd have

qi::rule<It, qi::space_type> a_whole, bunch_of, rules;

And then in your bunch_of rule you want to use differentRule which does have significant whitespace:

qi::rule<It, qi::blank_type> differentRule;

And you switch it like so:

bunch_of = "stuff" >> '{' >> qi::skip(qi::blank) [ differentRule ] >> '}';

That's all. Of course, if you wanted NO skipping at all inside differentRule you could have used qi::lexeme[] as always, or indeed just have dropped the skipper from the rule's declaration:

// implicit lexeme
qi::rule<It> differentRule;

Background

All of the above are detailed in this more broad answer: Boost spirit skipper issues

How can I parse list with newline in boost::spirit::qi?

1 Answers1

Different Skippers For Different Rules

Background