Boost Spirit X3: “Attribute does not have the expected size” redux

Question

Something seems to have changed in spirit::x3 that breaks my fancy little asdl parser. It worked just fine when I ported it over from Qi (after the initial x3 bugfixes made their way into fedora) but now fails with:

/usr/include/boost/spirit/home/x3/operator/detail/sequence.hpp:143:9: error: static assertion failed: Attribute does not have the expected size.

#include <boost/spirit/home/x3.hpp>
#include <boost/fusion/include/adapt_struct.hpp>

#include "asdl.h"

typedef boost::variant<asdl::ast::Product, asdl::ast::Sum> type_variant;
typedef std::vector<asdl::ast::Field> field_vec;

BOOST_FUSION_ADAPT_STRUCT(
    asdl::ast::Module,
    (std::string, id)
    (std::vector<asdl::ast::Type>, dfns))

BOOST_FUSION_ADAPT_STRUCT(
    asdl::ast::Type,
    (std::string, name)
    (type_variant, value))

BOOST_FUSION_ADAPT_STRUCT(
    asdl::ast::Field,
    (std::string, type)
    (boost::optional<char>, flag)
    (boost::optional<std::string>, name))

BOOST_FUSION_ADAPT_STRUCT(
    asdl::ast::Constructor,
    (std::string, name)
    (boost::optional<field_vec>, fields))

BOOST_FUSION_ADAPT_STRUCT(
    asdl::ast::Sum,
    (std::vector<asdl::ast::Constructor>, types)
    (boost::optional<field_vec>, attributes))

BOOST_FUSION_ADAPT_STRUCT(
    asdl::ast::Product,
    (field_vec, fields))

namespace asdl
{

namespace x3 = boost::spirit::x3;
namespace ascii = boost::spirit::x3::ascii;

x3::rule<class module,      ast::Module     > const module      = "module";
x3::rule<class definitions, ast::Type       > const definitions = "definitions";
x3::rule<class type,        type_variant    > const type        = "type";
x3::rule<class fields,      field_vec       > const fields      = "fields";
x3::rule<class field,       ast::Field      > const field       = "field";
x3::rule<class product,     ast::Product    > const product     = "product";
x3::rule<class sum,         ast::Sum        > const sum         = "sum";
x3::rule<class constructor, ast::Constructor> const constructor = "constructor";

auto const alpha             = '_' | ascii::upper | ascii::lower;
auto const alpha_num         = alpha | ascii::digit;
auto const typ_id            = x3::lexeme[ascii::lower >> *alpha_num];
auto const con_id            = x3::lexeme[ascii::upper >> *alpha_num];
auto const id                = typ_id | con_id;

auto const module_def        = "module" >> id >> '{' >> *definitions >> '}';
auto const definitions_def   = typ_id >> '=' >> type;
auto const type_def          = product | sum;
auto const product_def       = fields;
auto const sum_def           = constructor % '|' >> -("attributes" >> fields);
auto const constructor_def   = con_id >> -fields;
auto const fields_def        = '(' >> field % ',' >> ')';
auto const field_def         = typ_id >> -(ascii::char_('?') | ascii::char_('*')) >> -id;

BOOST_SPIRIT_DEFINE(module, definitions, type, product, sum, constructor, fields, field);


bool parse(const std::string &input, ast::Module &mod) {
    std::string::const_iterator iter = input.begin();
    std::string::const_iterator end = input.end();

    return (x3::phrase_parse(iter, end, module, ascii::space, mod) && iter == end);
}

} //namespace asdl

Note: BOOST_FUSION_ADAPT_STRUCT is a straight wrapper around bare structs with exactly the same fields.

I searched all (2) other references to "Attribute does not have the expected size" but don't quite understand the problem enough to even begin to fix the code -- part of the problem is the program generates the ast that it parses into (eat your own dogfood as they say) so without knowing why the assertion fails I don't know which to modify, the parser or the code that outputs the ast structs.

And here I thought that yak was done shaved...

[This](https://stackoverflow.com/questions/41101915/boost-spirit-x3-attribute-does-not-have-the-expected-size-but-why-does-it-ca) seems related, or at least gives some additional information about that particular static_assert — Scott Smith, Apr 20 '18 at 02:01
[Source code for Sequence.hpp, which contains the static_assert](https://www.boost.org/doc/libs/develop/boost/spirit/home/x3/operator/detail/sequence.hpp) — Scott Smith, Apr 20 '18 at 02:03
[Another SO post that may be related](https://stackoverflow.com/questions/38213729/spirit-x3-semantic-action-makes-compilation-fails-with-attribute-does-not-have) — Scott Smith, Apr 20 '18 at 02:05

score 4 · Accepted Answer · answered Apr 20 '18 at 08:53

Like in the other linked answers, the attribute propagation/compatibility rules aren't the same as in Qi.

Sometimes this is surprising, sometimes it might be a bug.

YOu can always make the attribute type explicit by wrapping in a rule. Almost all of the time this fixes the bug.

Note: in Boost 1.66.0 there is a regression that was fixed for later versions: How to make a recursive rule in boost spirit x3 in VS2017

In your case, the id rule don't magically assign to a string. Help it a bit:

auto const typ_id          = as<std::string>(x3::lexeme[ascii::lower >> *alpha_num]);
auto const con_id          = as<std::string>(x3::lexeme[ascii::upper >> *alpha_num]);
auto const id              = typ_id | con_id;

Where as is a simple helper:

template <typename T> static auto as = [](auto p) { return x3::rule<struct tag, T> {"as"} = p; };

Demo

Note that the code can be a bit simpler:

Live On Coliru

#include <boost/spirit/home/x3.hpp>
#include <boost/fusion/include/adapt_struct.hpp>

namespace asdl { namespace ast {
    struct Field {
        std::string type;
        boost::optional<char> flag;
        boost::optional<std::string> name;
    };

    typedef std::vector<asdl::ast::Field> field_vec;

    struct Constructor {
        std::string name;
        boost::optional<field_vec> fields;
    };

    struct Sum {
        std::vector<Constructor> types;
        boost::optional<field_vec> attributes;
    };

    struct Product {
        field_vec fields;
    };

    typedef boost::variant<Product, Sum> type_variant;

    struct Type {
        std::string name;
        type_variant value;
    };

    struct Module {
        std::string id;
        std::vector<Type> dfns;
    };
} }

BOOST_FUSION_ADAPT_STRUCT(asdl::ast::Module, id, dfns)
BOOST_FUSION_ADAPT_STRUCT(asdl::ast::Type, name, value)
BOOST_FUSION_ADAPT_STRUCT(asdl::ast::Field, type, flag, name)
BOOST_FUSION_ADAPT_STRUCT(asdl::ast::Constructor, name, fields)
BOOST_FUSION_ADAPT_STRUCT(asdl::ast::Sum, types, attributes)
BOOST_FUSION_ADAPT_STRUCT(asdl::ast::Product, fields)

namespace asdl
{
    namespace x3    = boost::spirit::x3;
    namespace ascii = boost::spirit::x3::ascii;

    using ast::type_variant;
    using ast::field_vec;

    x3::rule<class module,      ast::Module     > const module      = "module";
    x3::rule<class definitions, ast::Type       > const definitions = "definitions";
    x3::rule<class type,        type_variant    > const type        = "type";
    x3::rule<class fields,      field_vec       > const fields      = "fields";
    x3::rule<class field,       ast::Field      > const field       = "field";
    x3::rule<class product,     ast::Product    > const product     = "product";
    x3::rule<class sum,         ast::Sum        > const sum         = "sum";
    x3::rule<class constructor, ast::Constructor> const constructor = "constructor";

    template <typename T> static auto as = [](auto p) { return x3::rule<struct tag, T> {"as"} = p; };

    auto const alpha           = '_' | ascii::upper | ascii::lower;
    auto const alpha_num       = alpha | ascii::digit;
    auto const typ_id          = as<std::string>(x3::lexeme[ascii::lower >> *alpha_num]);
    auto const con_id          = as<std::string>(x3::lexeme[ascii::upper >> *alpha_num]);
    auto const id              = typ_id | con_id;

    auto const module_def      = "module" >> id >> '{' >> *definitions >> '}';
    auto const definitions_def = typ_id >> '=' >> type;
    auto const type_def        = product | sum;
    auto const product_def     = fields;
    auto const sum_def         = constructor % '|' >> -("attributes" >> fields);
    auto const constructor_def = con_id >> -fields;
    auto const fields_def      = '(' >> field % ',' >> ')';
    auto const field_def       = typ_id >> -(ascii::char_('?') | ascii::char_('*')) >> -id;

    BOOST_SPIRIT_DEFINE(module, definitions, type, product, sum, constructor, fields, field)

    bool parse(const std::string &input, ast::Module &mod) {
        std::string::const_iterator iter = input.begin();
        std::string::const_iterator end = input.end();

        return x3::phrase_parse(iter, end, module >> x3::eoi, ascii::space, mod);
    }
} //namespace asdl

int main() {
    asdl::ast::Module mod;
    asdl::parse("", mod);
}

Since you're not using recursive rules, there's no actual need for _def and BOOST_SPIRIT_DEFINE etc:

Live On Coliru

namespace x3    = boost::spirit::x3;
namespace ascii = boost::spirit::x3::ascii;

using ast::type_variant;
using ast::field_vec;


template <typename T> static auto as = [](auto p) { return x3::rule<struct tag, T> {"as"} = p; };

auto const alpha     = '_' | ascii::upper | ascii::lower;
auto const alpha_num = alpha | ascii::digit;
auto const typ_id    = as<std::string>(x3::lexeme[ascii::lower >> *alpha_num]);
auto const con_id    = as<std::string>(x3::lexeme[ascii::upper >> *alpha_num]);
auto const id        = typ_id | con_id;

auto const field
    = x3::rule<class constructor, ast::Field> {"field"}
    = as<ast::Field>(typ_id >> -(ascii::char_('?') | ascii::char_('*')) >> -id);
auto const fields
    = x3::rule<class sum, field_vec>{ "fields" }
    = '(' >> field % ',' >> ')';
auto const constructor
    = x3::rule<class product, ast::Constructor>{ "constructor" }
    = as<ast::Constructor>(con_id >> -fields);
auto const sum
    = x3::rule<class field, ast::Sum>{ "sum" }
    = as<ast::Sum>(constructor % '|' >> -("attributes" >> fields));
auto const product
    = x3::rule<class fields, ast::Product>{ "product" }
    = as<ast::Product>(fields);
auto const type
    = x3::rule<class type, type_variant>{ "type" }
    = product | sum;
auto const definitions
    = x3::rule<class definitions, ast::Type>{ "definitions" }
    = as<ast::Type>(typ_id >> '=' >> type);
auto const module
    = x3::rule<class module, ast::Module>{ "module" }
    = "module" >> id >> '{' >> *definitions >> '}';

bool parse(const std::string &input, ast::Module &mod) {
    std::string::const_iterator iter = input.begin();
    std::string::const_iterator end = input.end();

    return x3::phrase_parse(iter, end, module >> x3::eoi, ascii::space, mod);
}

If I understand this correctly the synthesized attribute needs to be "cast" to the rule attribute -- which is either a bug or changed behavior in a more recent version than the one I originally ported this over from Qi? — Uncle Entity, Apr 20 '18 at 17:12
Clearly it is changed behaviour. If you happen to have the exact version that allowed your initial X3 port to compile, I could be tempted to see whether the change is purposely or an accident. I've seen other cases where I'd rather have other attribute propagation, so finding a bug might improve my own life a bit too :) — sehe, Apr 20 '18 at 18:12

Boost Spirit X3: “Attribute does not have the expected size” redux

1 Answers1

Demo

Linked