Boost.Spirit.Qi: How to return attributes with Nabialek trick

Question

Following several tutorials (e.g. http://boost-spirit.com/home/articles/qi-example/nabialek-trick/) I want to use the Nabialek trick to have a dynamic parser. Parsing already works fine, but I don't get the attributes transported. Explanations like https://stackoverflow.com/a/9109972/2524462 suggest, that attributes should be possible but not arguments.

This is just a small example parsing a string and a number into a struct. It is just for showcasing my problem; this method should be used in a larger system later on, where the dynamic parser is really needed.

Question: How do I transport attributes with the Nabialek trick?

I'm not an spirit expert, so please bear with me. I'm using gcc 4.8.1 and boost 1.54.

#define BOOST_SPIRIT_DEBUG
#define BOOST_SPIRIT_USE_PHOENIX_V3
#include <boost/fusion/adapted/struct.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>

namespace qi = boost::spirit::qi;
namespace phx = boost::phoenix;

//------------------------------------------------------------------------------
// Data structure
struct myline {
  myline()
      : _n(0), _s("") {
  }

  myline(int n, std::string s)
      : _n(n), _s(s) {
  }

  void set(int n, std::string s) {
    _n = n;
    _s = s;
  }

  int _n;
  std::string _s;
};

BOOST_FUSION_ADAPT_STRUCT(::myline, (int, _n) (std::string, _s))

//------------------------------------------------------------------------------
// Parser grammar
template<typename It, typename Skipper = qi::space_type>
struct parser: qi::grammar<It, myline(), Skipper> {
  parser()
      : parser::base_type(start) {
    using namespace qi;

    start = line;

    string %= qi::lexeme["'" >> *~qi::char_("'") >> "'"];

    one = (string >> "@" >> qi::int_)[_val = phx::construct<myline>(_2, _1)];
    two = (qi::int_ >> "@" >> string);

    keyword.add("one", &one)("two", &two);

    line = keyword[_a = _1] >> qi::lazy(*_a);

    on_error<fail>(
        start,
        std::cout << phx::val("Error! Expecting ") << _4
        << phx::val(" here: \"") << phx::construct<std::string>(_3, _2)
        << phx::val("\"\n"));

    BOOST_SPIRIT_DEBUG_NODES((start)(line)(one)(two))
  }

private:
  template<typename Attr> using Rule = qi::rule<It, Attr(), Skipper>;

  Rule<myline> start, one, two;
  qi::rule<It, myline, Skipper, qi::locals<Rule<myline>*> > line;

  Rule<std::string> string;

  qi::symbols<char, Rule<myline>*> keyword;
};

//------------------------------------------------------------------------------
int main() {
  for (const std::string input : std::vector<std::string> { "one 'test'@1",
                                                            "two 2@'test'" }) {
    auto f(std::begin(input)), l(std::end(input));
    const static parser<decltype(f)> p;

    myline parsed_script;
    bool ok = qi::phrase_parse(f, l, p, qi::space, parsed_script);

    if (!ok) {
      std::cout << "invalid input\n";
    }

    std::cout << parsed_script._n << ": " << parsed_script._s << std::endl;

    if (f != l) {
      std::cout << "unparsed: '" << std::string(f, l) << "'" << std::endl;
    }
  }
}

Parsing result:

<start>
  <try>one 'test'@1</try>
  <line>
    <try>one 'test'@1</try>
    <one>
      <try> 'test'@1</try>
      <success></success>
      <attributes>[[1, [t, e, s, t]]]</attributes>
    </one>
    <success></success>
    <attributes>[]</attributes><locals>(0x43b0e0)</locals>
  </line>
  <success></success>
  <attributes>[[0, []]]</attributes>
</start>
<start>
  <try>two 2@'test'</try>
  <line>
    <try>two 2@'test'</try>
    <two>
      <try> 2@'test'</try>
      <success></success>
      <attributes>[[2, [t, e, s, t]]]</attributes>
    </two>
    <success></success>
    <attributes>[]</attributes><locals>(0x43b110)</locals>
  </line>
  <success></success>
  <attributes>[[0, []]]</attributes>
</start>

score 17 · Accepted Answer · edited May 23 '17 at 11:46

You have been paying a lot of attention in Spirit Class :)

There were a number of issues:

the attribute declaration of the line rule was wrong:

qi::rule<It, myline, Skipper, qi::locals<Rule<myline>*> > line;

needs to be

qi::rule<It, myline(), Skipper, qi::locals<Rule<myline>*> > line;

Automatic attribute propagation is inhibited in the presence of semantic actions. See a recent answer for more information: Boost.spirit: parsing number char and string. So you need to explicitely engage Spirit's auto-rule behaviour using %=:
```
line = keyword[_a = _1] >> qi::lazy(*_a);
```
nees to be
```
// NOTE the %=
line %= omit [ keyword[_a = _1] ] >> qi::lazy(*_a);
```
Notes:
- the %= can go on the string rule (no semantic actions imply automatic attribute propagation)
- we need to explicitely omit[] the result of the keyword match, because we can't really assign the Rule<>* to our myline attribute

Here's the fixed version:

#define BOOST_SPIRIT_DEBUG
#define BOOST_SPIRIT_USE_PHOENIX_V3
#include <boost/fusion/adapted/struct.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>

namespace qi = boost::spirit::qi;
namespace phx = boost::phoenix;

//------------------------------------------------------------------------------
// Data structure
struct myline {
  myline()
      : _n(0), _s("") {
  }

  myline(int n, std::string s)
      : _n(n), _s(s) {
  }

  void set(int n, std::string s) {
    _n = n;
    _s = s;
  }

  int _n;
  std::string _s;
};

BOOST_FUSION_ADAPT_STRUCT(::myline, (int, _n) (std::string, _s))

//------------------------------------------------------------------------------
// Parser grammar
template<typename It, typename Skipper = qi::space_type>
struct parser: qi::grammar<It, myline(), Skipper> {
  parser()
      : parser::base_type(start) {
    using namespace qi;

    start  = line;

    string = qi::lexeme["'" >> *~qi::char_("'") >> "'"];

    one    = (string >> "@" >> qi::int_) [_val           = phx::construct<myline>(_2, _1)];
    two    = (qi::int_ >> "@" >> string);

    keyword.add("one", &one)("two", &two);

    // NOTE the %=
    line %= omit [ keyword[_a = _1] ] >> qi::lazy(*_a);

    on_error<fail>(
        start,
        std::cout << phx::val("Error! Expecting ") << _4
        << phx::val(" here: \"") << phx::construct<std::string>(_3, _2)
        << phx::val("\"\n"));

    BOOST_SPIRIT_DEBUG_NODES((start)(line)(one)(two))
  }

private:
  template<typename Attr> using Rule = qi::rule<It, Attr(), Skipper>;

  Rule<myline> start, one, two;
  qi::rule<It, myline(), Skipper, qi::locals<Rule<myline>* > > line;

  Rule<std::string> string;

  qi::symbols<char, Rule<myline>* > keyword;
};

//------------------------------------------------------------------------------
int main() {
  for (const std::string input : std::vector<std::string> { "one 'test1'@1",
                                                            "two 2@'test2'" }) {
    auto f(std::begin(input)), l(std::end(input));
    const static parser<decltype(f)> p;

    myline parsed_script;
    bool ok = qi::phrase_parse(f, l, p, qi::space, parsed_script);

    if (!ok) {
      std::cout << "invalid input\n";
    }

    std::cout << parsed_script._n << ": " << parsed_script._s << std::endl;

    if (f != l) {
      std::cout << "unparsed: '" << std::string(f, l) << "'" << std::endl;
    }
  }
}

Prints:

<start>
  <try>one 'test1'@1</try>
  <line>
    <try>one 'test1'@1</try>
    <one>
      <try> 'test1'@1</try>
      <success></success>
      <attributes>[[1, [t, e, s, t, 1]]]</attributes>
    </one>
    <success></success>
    <attributes>[[1, [t, e, s, t, 1]]]</attributes><locals>(0x6386c0)</locals>
  </line>
  <success></success>
  <attributes>[[1, [t, e, s, t, 1]]]</attributes>
</start>
1: test1
<start>
  <try>two 2@'test2'</try>
  <line>
    <try>two 2@'test2'</try>
    <two>
      <try> 2@'test2'</try>
      <success></success>
      <attributes>[[2, [t, e, s, t, 2]]]</attributes>
    </two>
    <success></success>
    <attributes>[[2, [t, e, s, t, 2]]]</attributes><locals>(0x6386f0)</locals>
  </line>
  <success></success>
  <attributes>[[2, [t, e, s, t, 2]]]</attributes>
</start>
2: test2

See the fixed sample **[live on Coliru](http://coliru.stacked-crooked.com/view?id=339ded0e400164a92e13f5cb65e2e54e-414c51b0c3fcfe921b347f2307ac5c70)** — sehe, Jul 18 '13 at 08:59
Thank you very much, works very well. Seems I still need several more lessons ;-) — Mike M, Jul 18 '13 at 09:00
too bad there are too few Boost.Spirit users out there for more upvotes. You will get them when you reincarnate :-) — TemplateRex, Jul 18 '13 at 11:01
TemplateRex, don't mention that when NicolBolas can hear you. I'm pretty sure he'll [demand a proof of your claim](http://meta.stackexchange.com/questions/173045/wiping-votes-on-deletion-of-highly-active-accounts-865-points-on-user-was-rem/173076#comment536206_173076) :). Also, this is already extraordinarily well-upvoted, so I don't complain @TemplateRex — sehe, Jul 18 '13 at 12:19
@sehe at some point I am gonna learn Boost.Spirit, after all, after learning quantum mechanics and template metaprogramming, how hard can it be? ;-) — TemplateRex, Jul 18 '13 at 12:28
You'll be right at home. I've seen your TMP. Better than mine :) — sehe, Jul 18 '13 at 12:37

Boost.Spirit.Qi: How to return attributes with Nabialek trick

1 Answers1

Linked