0

Some time ago I wrote spirit parsing code in windows, which just worked fine. Now I am trying to build it on Ubuntu, but c++ (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) fails with some error messages, and I don't have the slightest bit of a glue what to do (line 161 is marked by the comment "//LINE 161!!!!"):

#include <string>
#include <map>
#include <boost/config/warning_disable.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/qi_hold.hpp>
#include <boost/spirit/include/qi_omit.hpp>
#include <boost/spirit/include/phoenix_core.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <boost/spirit/include/phoenix_stl.hpp>
#include <boost/spirit/include/phoenix_fusion.hpp>
#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/fusion/include/io.hpp>
#include <boost/variant/recursive_variant.hpp>
#include <boost/foreach.hpp>
#include <exception>
#include <vector>
using namespace std;
using namespace boost;
using boost::spirit::qi::phrase_parse;
using boost::spirit::qi::char_;
using boost::spirit::qi::double_;
using boost::spirit::qi::eps;
using boost::spirit::qi::lit;
using boost::spirit::qi::_1;
using boost::spirit::qi::grammar;
using boost::spirit::qi::lexeme;
using boost::spirit::qi::symbols;
using boost::spirit::qi::rule;
using boost::spirit::qi::hold;
using boost::spirit::qi::omit;
using boost::spirit::_val;
using boost::spirit::ascii::space;
using boost::spirit::ascii::space_type;
using boost::phoenix::ref;
using boost::phoenix::push_back;
using boost::phoenix::at_c;

namespace sx {

    namespace parserInternal {

        /**
         * types of _XNODE_
         */
        enum _XTYPE_ {
            _XTEXT_ ,
            _XTAG_ ,
            _S_ATTRIB_,
            _R_ATTRIB_
        };

        /**
         * structure for boost spirit
         * parsetree datastructure
         */
        struct _XNODE_ {

            /**
             * type of node
             */
            int type;

            /**
             * data of XText
             */
            string text;

            /**
             * data of XTag
             */
            string name;
            vector<_XNODE_> nodes;

            /**
             * data of string attribute
             */
            string strID;
            string strAttrib;

            /**
             * data of real attribute
             */
            string rID;
            double rAttrib;

            /**
             * bug fix - stop parser from
             * taking characters it shouldn't
             * by assigning eat with the useless
             * string sequences
             */
            string eat;

        };

    }

}

BOOST_FUSION_ADAPT_STRUCT (
    sx::parserInternal::_XNODE_ ,
    (int                                        ,type)      //  0
    (std::string                                ,text)      //  1 - XText
    (std::string                                ,name)      //  2 - XTag
    (std::vector<sx::parserInternal::_XNODE_>   ,nodes)     //  3 - XTag
    (std::string                                ,strID)     //  4 - str. attrib
    (std::string                                ,strAttrib) //  5 - str. attrib
    (std::string                                ,rID)       //  6 - r. attrib
    (double                                     ,rAttrib)   //  7 - r. attrib
    (std::string                                ,eat)       //  8 - bug fix
)

namespace sx {

    namespace parserInternal {

        /**
         * filters comments out of the text
         */
        struct SXFilter: public grammar<string::iterator,string()> {

            /**
             * start rule
             */
            rule<string::iterator,string()> start;

            /**
             * recognizes a sequence starting with //, and anything
             * ending with newline
             */
            rule<string::iterator,string()> skipSmallComment;

            /**
             * recognizes a sequence starting with /* and anything
             * ending with the two characters '*' and '/' in sequence
             */
            rule<string::iterator,string()> skipLargeComment;

            /**
             * recognizes newline
             */
            rule<string::iterator,string()> separator;

            /**
             * recognizes any text not containing the char sequences
             * /* amd //
             */
            rule<string::iterator,string()> acceptable;

            SXFilter(): SXFilter::base_type(start) {
                separator %= lexeme[(char_('\n') | char_('\r\n'))];
                acceptable %= *lexeme[char_ - lit("/*") - lit("//")];
                skipLargeComment %= lit("/*") >> *lexeme[char_ - lit("*/")] >> lit("*/");
                skipSmallComment %= lit ("//") >> *lexeme[char_ - separator];
                start %= eps >> acceptable >>
                    *(
                        (
                            omit[skipSmallComment]
                            | omit[skipLargeComment]
                        ) >> acceptable
                    )
//LINE 161!!!!
                    ;
            }

        };

        /**
         * grammar for the parser
         */
        struct XGrammar: public grammar<string::iterator,_XNODE_(),space_type> {

            /**
             * a tag
             */
            rule<string::iterator,_XNODE_(),space_type> tag;

            /**
             * child nodes of a tag
             */
            rule<string::iterator,vector<_XNODE_>(),space_type> nodelist;

            /**
             * identifyer - starts with a letter in a-zA-Z_:. , and can be
             * continued by a-zA-Z_0-9:. , must have at least one letter
             */
            rule<string::iterator,string()> identifyer;

            /**
             * any char sequence without the letter " of any length
             * bordered by the letter "
             */
            rule<string::iterator,string()> textdata;

            /**
             * attribute assigned with string value
             */
            rule<string::iterator,_XNODE_(),space_type> strAttrib;

            /**
             * attribute assigned with double value
             */
            rule<string::iterator,_XNODE_(),space_type> realAttrib;

            /**
             * simply textdata returning _XNODE_
             */
            rule<string::iterator,_XNODE_(),space_type> textNode;

            /**
             * constructor, makes tag to the node's root
             */
            XGrammar(): XGrammar::base_type(tag) {
                identifyer %= lexeme[char_("a-zA-Z_:.") >> *( char_("0-9a-zA-Z_:.") )];
                textdata %= lexeme['"' >> *(char_ - '"') >> '"'];
                strAttrib %= 
                    identifyer[at_c<4>(_val) = _1] >> char_('=') >> 
                    textdata[at_c<5>(_val) = _1] >> char_(';')[at_c<0>(_val) = _S_ATTRIB_];
                realAttrib %=
                    identifyer[at_c<6>(_val) = _1] >> char_('=') >>
                    double_[at_c<7>(_val) = _1] >> char_(';')[at_c<0>(_val) = _R_ATTRIB_];
                textNode %= textdata[at_c<1>(_val) = _1][at_c<0>(_val) = _XTEXT_];
                nodelist %= eps >>
                    *(
                        tag
                        | strAttrib
                        | realAttrib
                        | textNode
                    )
                    ;
                tag %= eps >>
                    char_('(') >> identifyer[at_c<2>(_val) = _1] >> char_(')')[at_c<8>(_val) = _1] >>
                    (
                        char_('{') >>
                        nodelist[at_c<3>(_val) = _1] >>
                        char_('}')
                    | eps
                    )[at_c<0>(_val) = _XTAG_]
                    ;
            }

        };

        void parseSXdata1(const string &data, string &output) {
            string filterable = data;
            string::iterator iter1 = filterable.begin();
            string::iterator iter2 = filterable.end();
            SXFilter filter;
            bool parsed = phrase_parse(
                iter1,
                iter2,
                filter,
                space,
                output
                );
            if(!parsed || iter1 != iter2) {
                throw std::exception();
            }
        }

        void parseSXdata2(string &data, parserInternal::_XNODE_ &output) {
            string::iterator iter1 = data.begin();
            string::iterator iter2 = data.end();
            XGrammar grammar;
            bool parsed = phrase_parse(
                iter1,
                iter2,
                grammar,
                space,
                output
                );
            if(!parsed || iter1 != iter2) {
                throw std::exception();
            }
        }

    }
}

int main(int argc, char **argv) {
    string data = 
        "(testsx) {\n"
        "   (test) {\"hello world\"}\n"
        "   (test2) {\n"
        "       attrib1 = 123;\n"
        "       attrib2 = \"hey\";\n"
        "   }\n"
        "}"
        ;

    string iterable;
    sx::parserInternal::_XNODE_ output; //root of parsetree
    sx::parserInternal::parseSXdata1(data,iterable);
    sx::parserInternal::parseSXdata2(iterable,output);

    return 0;
}

I successfully compiled that code in MSVC 2008 using the boost libraries 1.47. But in Ubuntu gcc 4.6.3 produces the following error messages:

Main.cpp:151:46: warning: multi-character character constant [-Wmultichar]
In file included from /usr/include/boost/spirit/home/qi/auxiliary/attr.hpp:18:0,
             from /usr/include/boost/spirit/home/qi/auxiliary.hpp:19,
             from /usr/include/boost/spirit/home/qi.hpp:16,
             from /usr/include/boost/spirit/include/qi.hpp:16,
             from Main.cpp:4:
/usr/include/boost/spirit/home/qi/detail/assign_to.hpp: In static member function ‘static void boost::spirit::traits::assign_to_attribute_from_value<Attribute, T, Enable>::call(const T_&, Attribute&, mpl_::false_) [with T_ = std::basic_string<char>, Attribute = char, T = std::basic_string<char>, Enable = void, mpl_::false_ = mpl_::bool_<false>]’:
/usr/include/boost/spirit/home/qi/detail/assign_to.hpp:127:13:   instantiated from ‘static void boost::spirit::traits::assign_to_attribute_from_value<Attribute, T, Enable>::call(const T&, Attribute&) [with Attribute = char, T = std::basic_string<char>, Enable = void]’
/usr/include/boost/spirit/home/qi/detail/assign_to.hpp:234:13:   instantiated from ‘void boost::spirit::traits::detail::assign_to(const T&, Attribute&, P1, P2) [with T = std::basic_string<char>, Attribute = char, P1 = mpl_::bool_<false>, P2 = mpl_::bool_<true>]’
/usr/include/boost/spirit/home/qi/detail/assign_to.hpp:257:9:   instantiated from ‘void boost::spirit::traits::assign_to(const T&, Attribute&) [with T = std::basic_string<char>, Attribute = char]’
/usr/include/boost/spirit/home/qi/detail/attributes.hpp:26:13:   instantiated from ‘static void boost::spirit::qi::default_transform_attribute<Exposed, Transformed>::post(Exposed&, const Transformed&) [with Exposed = char, Transformed = std::basic_string<char>]’
/usr/include/boost/spirit/home/qi/detail/attributes.hpp:164:86:   instantiated from ‘void boost::spirit::traits::post_transform(Exposed&, const Transformed&) [with Exposed = char, Transformed = std::basic_string<char>]’
/usr/include/boost/spirit/home/qi/nonterminal/rule.hpp:281:21:   [ skipping 20 instantiation contexts ]
/usr/include/boost/spirit/home/qi/operator/sequence_base.hpp:123:50:   instantiated from ‘bool boost::spirit::qi::sequence_base<Derived, Elements>::parse(Iterator&, const Iterator&, Context&, const Skipper&, Attribute&) const [with Iterator = __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, Context = boost::spirit::context<boost::fusion::cons<std::basic_string<char>&, boost::fusion::nil>, boost::fusion::vector0<> >, Skipper = boost::spirit::unused_type, Attribute = std::basic_string<char>, Derived = boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::eps_parser, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::cons<boost::spirit::qi::kleene<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::alternative<boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::nil> > >, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::nil> > > >, boost::fusion::nil> > > >, Elements = boost::fusion::cons<boost::spirit::qi::eps_parser, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::cons<boost::spirit::qi::kleene<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::alternative<boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::nil> > >, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::nil> > > >, boost::fusion::nil> > >]’
/usr/include/boost/spirit/home/qi/nonterminal/detail/parser_binder.hpp:73:54:   instantiated from ‘bool boost::spirit::qi::detail::parser_binder<Parser, mpl_::bool_<true> >::operator()(Iterator&, const Iterator&, Context&, const Skipper&) const [with Iterator = __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, Skipper = boost::spirit::unused_type, Context = boost::spirit::context<boost::fusion::cons<std::basic_string<char>&, boost::fusion::nil>, boost::fusion::vector0<> >, Parser = boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::eps_parser, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::cons<boost::spirit::qi::kleene<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::alternative<boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::nil> > >, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::nil> > > >, boost::fusion::nil> > > >]’
/usr/include/boost/function/function_template.hpp:132:42:   instantiated from ‘static R boost::detail::function::function_obj_invoker4<FunctionObj, R, T0, T1, T2, T3>::invoke(boost::detail::function::function_buffer&, T0, T1, T2, T3) [with FunctionObj = boost::spirit::qi::detail::parser_binder<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::eps_parser, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::cons<boost::spirit::qi::kleene<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::alternative<boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::nil> > >, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::nil> > > >, boost::fusion::nil> > > >, mpl_::bool_<true> >, R = bool, T0 = __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >&, T1 = const __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >&, T2 = boost::spirit::context<boost::fusion::cons<std::basic_string<char>&, boost::fusion::nil>, boost::fusion::vector0<> >&, T3 = const boost::spirit::unused_type&]’
/usr/include/boost/function/function_template.hpp:913:60:   instantiated from ‘void boost::function4<R, T1, T2, T3, T4>::assign_to(Functor) [with Functor = boost::spirit::qi::detail::parser_binder<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::eps_parser, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::cons<boost::spirit::qi::kleene<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::alternative<boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::nil> > >, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::nil> > > >, boost::fusion::nil> > > >, mpl_::bool_<true> >, R = bool, T0 = __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >&, T1 = const __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >&, T2 = boost::spirit::context<boost::fusion::cons<std::basic_string<char>&, boost::fusion::nil>, boost::fusion::vector0<> >&, T3 = const boost::spirit::unused_type&]’
/usr/include/boost/function/function_template.hpp:722:7:   instantiated from ‘boost::function4<R, T1, T2, T3, T4>::function4(Functor, typename boost::enable_if_c<boost::type_traits::ice_not<boost::is_integral<Functor>::value>::value, int>::type) [with Functor = boost::spirit::qi::detail::parser_binder<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::eps_parser, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::cons<boost::spirit::qi::kleene<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::alternative<boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::nil> > >, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::nil> > > >, boost::fusion::nil> > > >, mpl_::bool_<true> >, R = bool, T0 = __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >&, T1 = const __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >&, T2 = boost::spirit::context<boost::fusion::cons<std::basic_string<char>&, boost::fusion::nil>, boost::fusion::vector0<> >&, T3 = const boost::spirit::unused_type&, typename boost::enable_if_c<boost::type_traits::ice_not<boost::is_integral<Functor>::value>::value, int>::type = int]’
/usr/include/boost/function/function_template.hpp:1064:16:   instantiated from ‘boost::function<R(T0, T1, T2, T3)>::function(Functor, typename boost::enable_if_c<boost::type_traits::ice_not<boost::is_integral<Functor>::value>::value, int>::type) [with Functor = boost::spirit::qi::detail::parser_binder<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::eps_parser, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::cons<boost::spirit::qi::kleene<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::alternative<boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::nil> > >, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::nil> > > >, boost::fusion::nil> > > >, mpl_::bool_<true> >, R = bool, T0 = __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >&, T1 = const __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >&, T2 = boost::spirit::context<boost::fusion::cons<std::basic_string<char>&, boost::fusion::nil>, boost::fusion::vector0<> >&, T3 = const boost::spirit::unused_type&, typename boost::enable_if_c<boost::type_traits::ice_not<boost::is_integral<Functor>::value>::value, int>::type = int]’
/usr/include/boost/function/function_template.hpp:1105:5:   instantiated from ‘typename boost::enable_if_c<boost::type_traits::ice_not<boost::is_integral<Functor>::value>::value, boost::function<R(T0, T1, T2, T3)>&>::type boost::function<R(T0, T1, T2, T3)>::operator=(Functor) [with Functor = boost::spirit::qi::detail::parser_binder<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::eps_parser, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::cons<boost::spirit::qi::kleene<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::alternative<boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::cons<boost::spirit::qi::omit_directive<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> > >, boost::fusion::nil> > >, boost::fusion::cons<boost::spirit::qi::reference<const boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type> >, boost::fusion::nil> > > >, boost::fusion::nil> > > >, mpl_::bool_<true> >, R = bool, T0 = __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >&, T1 = const __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >&, T2 = boost::spirit::context<boost::fusion::cons<std::basic_string<char>&, boost::fusion::nil>, boost::fusion::vector0<> >&, T3 = const boost::spirit::unused_type&, typename boost::enable_if_c<boost::type_traits::ice_not<boost::is_integral<Functor>::value>::value, boost::function<R(T0, T1, T2, T3)>&>::type = boost::function<bool(__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >&, const __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >&, boost::spirit::context<boost::fusion::cons<std::basic_string<char>&, boost::fusion::nil>, boost::fusion::vector0<> >&, const boost::spirit::unused_type&)>&]’
/usr/include/boost/spirit/home/qi/nonterminal/rule.hpp:214:13:   instantiated from ‘boost::spirit::qi::rule<Iterator, T1, T2, T3, T4>& boost::spirit::qi::operator%=(boost::spirit::qi::rule<Iterator, T1, T2, T3, T4>&, const Expr&) [with Expr = boost::proto::exprns_::expr<boost::proto::tag::shift_right, boost::proto::argsns_::list2<const boost::proto::exprns_::expr<boost::proto::tag::shift_right, boost::proto::argsns_::list2<const boost::spirit::terminal<boost::spirit::tag::eps>&, boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type>&>, 2l>&, const boost::proto::exprns_::expr<boost::proto::tag::dereference, boost::proto::argsns_::list1<const boost::proto::exprns_::expr<boost::proto::tag::shift_right, boost::proto::argsns_::list2<const boost::proto::exprns_::expr<boost::proto::tag::bitwise_or, boost::proto::argsns_::list2<const boost::proto::exprns_::expr<boost::proto::tag::subscript, boost::proto::argsns_::list2<const boost::proto::exprns_::expr<boost::proto::tag::terminal, boost::proto::argsns_::term<boost::spirit::tag::omit>, 0l>&, boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type>&>, 2l>&, const boost::proto::exprns_::expr<boost::proto::tag::subscript, boost::proto::argsns_::list2<const boost::proto::exprns_::expr<boost::proto::tag::terminal, boost::proto::argsns_::term<boost::spirit::tag::omit>, 0l>&, boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type>&>, 2l>&>, 2l>&, boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type>&>, 2l>&>, 1l>&>, 2l>, Iterator = __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, T1 = std::basic_string<char>(), T2 = boost::spirit::unused_type, T3 = boost::spirit::unused_type, T4 = boost::spirit::unused_type, boost::spirit::qi::rule<Iterator, T1, T2, T3, T4> = boost::spirit::qi::rule<__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, std::basic_string<char>(), boost::spirit::unused_type, boost::spirit::unused_type, boost::spirit::unused_type>]’
Main.cpp:161:6:   instantiated from here
/usr/include/boost/spirit/home/qi/detail/assign_to.hpp:109:13: error: invalid static_cast from type ‘const std::basic_string<char>’ to type ‘char’
make: *** [part1] Error 1

I hope someone has an idea what went wrong.

user1882245
  • 101
  • 1
  • 4

2 Answers2

2

So let me post the big refactoring in a separate answer.

It took a little more than 30 minutes. Actually, a little over an hour. It's basically a rewrite.

It comes in at 171 lines of code (including debug and sample test). See it Live On Coliru. Jumping to the meat, here is your entire grammar:

    identifier  = lexeme[ char_("a-zA-Z_:.") >> *char_("0-9a-zA-Z_:.") ];
    quoted_text = lexeme[ '"' >> *(char_ - '"') >> '"' ];

    strAttrib   = identifier >> '=' >> quoted_text >> ';';
    realAttrib  = identifier >> '=' >> double_     >> ';';
    textNode    = quoted_text;

    nodelist    = '{' >> *node >> '}';

    node        = tag
                | strAttrib
                | realAttrib
                | textNode
                ;

    tag         = '(' >> identifier >> ')' >> -nodelist;
                ;

    // allow only tags at root of parse tree
    start       = tag;

21 lines of clean grammar productions.
Are you impressed? I think you should be. This is the power of Boost Spirit at it's best. See also

Now you might wonder what the AST turned out to be:

namespace AST
{
    struct TextNode {
        std::string text;

        // single element structs don't work well with Fusion Adapation...
        TextNode(std::string text = "") : text(std::move(text)) {}
    };

    template <typename V> struct Attr_ {
        std::string ID;
        V           Attrib;

        Attr_(std::string ID = "", V Attrib = V())
            : ID(std::move(ID)), Attrib(std::move(Attrib))
        { }
    };

    typedef Attr_<std::string> StringAttribute;
    typedef Attr_<double>      RealAttribute;

    struct TagNode;

    typedef boost::variant<
        TextNode,
        boost::recursive_wrapper<TagNode>,
        StringAttribute,
        RealAttribute
    > Node;

    // recursive node
    struct TagNode {
        std::string       name;
        std::vector<Node> nodes;
    };
}

So, more code, but at 36 lines, roughly half of the original bits. More importantly:

  • The code expresses and reinforces the design (no SAttr members in a XTAG node, e.g.)
  • The nodes are smaller. Much smaller. On my box (gcc or clang optimized 64bit/32bit)

    sizeof(sx::parserInternal::XNODE_):      88/48 bytes (!!!)
    sizeof(sx::parserInternal::AST::Node):   24/16 bytes
    

    That's 3-fold improvement of storage efficiency. Regardless of the actual data. Now, with many non-tag nodes the saving may be even more due to absense of dynamic allocations for the vector.

  • But most importantly: zero-overhead attribute propagation. This is what

    • enabled us to cut Fusion and Phoenix out of the picture (compiletimes, anyone?)
    • enabled us to propagate all attributes without a single semantic action
  • It will also trivially allow us to operate on the tree, visiting it's nodes, transforming it, etc. I'm getting way beyond the scope here, but let me show a small sample, that shows you how you could recursively transform the new AST::Node style tree into your "legace" XNODE_ tree:

    struct BuildLegacyXNode : boost::static_visitor<XNODE_> {
        XNODE_ operator()(AST::TextNode        const& n) const { XNODE_ r; r.type = XTEXT_;    r.text = n.text;  return r; }
        XNODE_ operator()(AST::TagNode         const& n) const { XNODE_ r; r.type = XTAG_;     r.name = n.name;  r.nodes = xform(n.nodes); return r; }
        XNODE_ operator()(AST::StringAttribute const& n) const { XNODE_ r; r.type = S_ATTRIB_; r.strID = n.ID;   r.strAttrib = n.Attrib;     return r; }
        XNODE_ operator()(AST::RealAttribute   const& n) const { XNODE_ r; r.type = R_ATTRIB_; r.rID = n.ID;     r.rAttrib = n.Attrib;       return r; }
        XNODE_ operator()(AST::Node            const& n) const { return boost::apply_visitor(*this, n); }
      private:
        std::vector<XNODE_> xform(std::vector<AST::Node> const& n) const {
            std::vector<XNODE_> r(n.size());
            std::transform(n.begin(), n.end(), r.begin(), *this);
            return r;
        }
    };
    

As a bonus, here's a demo showing that working in c++03 mode on Coliru

using namespace sx::parserInternal;
const AST::Node output = parseSXdata2(data); // root of parsetree

BuildLegacyXNode transform;
XNODE_ legacy = transform(output);

std::cout << "Root tag is named '" << legacy.name << "' and has " << legacy.nodes.size() << " direct child nodes\n";

Output:

AST::Node: 24 bytes
XNODE_:    88 bytes
Root tag is named 'testsx' and has 2 direct child nodes

So, even the full code for transforming the new AST to the old AST doesn't even make up for the code that we saved in the rule definitions alone. And the compilation time is reduced with ~20%.

TL;DR

  1. Simplicity is key
  2. Simplicity comes with convention (in a good API framework)

Full Code

// #define BOOST_SPIRIT_DEBUG
#include <boost/config/warning_disable.hpp>
#include <boost/fusion/adapted/struct.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/fusion/include/io.hpp>
#include <boost/variant/recursive_variant.hpp>
#include <stdexcept>
#include <vector>
#include <string>
#include <map>

namespace qi  = boost::spirit::qi;
namespace phx = boost::phoenix;

namespace sx {
    namespace parserInternal {
        /**
         * structure for boost spirit
         * parsetree datastructure
         */
        namespace AST
        {
            struct TextNode {
                std::string text;

                // single element structs don't work well with Fusion Adapation...
                TextNode(std::string const& text = "") : text(text) {}
            };

            template <typename V> struct Attr_ {
                std::string ID;
                V           Attrib;

                Attr_(std::string const& ID = "", V const& Attrib = V())
                    : ID(ID), Attrib(Attrib)
                { }
            };

            typedef Attr_<std::string> StringAttribute;
            typedef Attr_<double>      RealAttribute;

            struct TagNode;

            typedef boost::variant<
                TextNode,
                boost::recursive_wrapper<TagNode>,
                StringAttribute,
                RealAttribute
            > Node;

            // recursive node
            struct TagNode {
                std::string       name;
                std::vector<Node> nodes;
            };
        }

        /**
         * example of how you can easily visit nodes
         */
        enum XTYPE_ { XTEXT_ , XTAG_ , S_ATTRIB_, R_ATTRIB_ };
        struct XNODE_ {
            XTYPE_ type; // type code
            std::string text;                            // TextNode
            std::string name; std::vector<XNODE_> nodes; // TagNode
            std::string strID, strAttrib;                // StringAttribute
            std::string rID; double rAttrib;             // RealAttribute
            std::string eat;
        };
        struct BuildLegacyXNode : boost::static_visitor<XNODE_> {
            XNODE_ operator()(AST::TextNode        const& n) const { XNODE_ r; r.type = XTEXT_;    r.text = n.text;  return r; }
            XNODE_ operator()(AST::TagNode         const& n) const { XNODE_ r; r.type = XTAG_;     r.name = n.name;  r.nodes = xform(n.nodes);   return r; }
            XNODE_ operator()(AST::StringAttribute const& n) const { XNODE_ r; r.type = S_ATTRIB_; r.strID = n.ID;   r.strAttrib = n.Attrib;     return r; }
            XNODE_ operator()(AST::RealAttribute   const& n) const { XNODE_ r; r.type = R_ATTRIB_; r.rID = n.ID;     r.rAttrib = n.Attrib;       return r; }
            XNODE_ operator()(AST::Node            const& n) const { return boost::apply_visitor(*this, n); }
          private:
            std::vector<XNODE_> xform(std::vector<AST::Node> const& n) const {
                std::vector<XNODE_> r(n.size());
                std::transform(n.begin(), n.end(), r.begin(), *this);
                return r;
            }
        };
    }
}

BOOST_FUSION_ADAPT_STRUCT(sx::parserInternal::AST::TagNode, (std::string, name) (std::vector<sx::parserInternal::AST::Node>, nodes))
BOOST_FUSION_ADAPT_STRUCT(sx::parserInternal::AST::TextNode, (std::string, text))
BOOST_FUSION_ADAPT_TPL_STRUCT((V), (sx::parserInternal::AST::Attr_)(V), (std::string, ID)(V, Attrib))

namespace sx {
    namespace parserInternal {
        /**
         * grammar for the parser
         */
        template <
            typename It = std::string::const_iterator,
            typename Skipper = qi::rule<It>
        >
        struct XGrammar: qi::grammar<It, AST::Node(), Skipper> {

            qi::rule<It, std::vector<AST::Node>(), Skipper> nodelist;
            qi::rule<It, AST::Node(),              Skipper> node, start;
            qi::rule<It, AST::TagNode(),           Skipper> tag;
            qi::rule<It, AST::TextNode(),          Skipper> textNode;
            qi::rule<It, AST::StringAttribute(),   Skipper> strAttrib;
            qi::rule<It, AST::RealAttribute(),     Skipper> realAttrib;

            // natural lexemes (using `lexeme` there is a bit redundant):
            qi::rule<It, std::string()> identifier;
            qi::rule<It, std::string()> quoted_text;

            /**
             * constructor, makes tag to the node's root
             */
            XGrammar(): XGrammar::base_type(start) {

                using namespace qi;

                identifier  = lexeme[ char_("a-zA-Z_:.") >> *char_("0-9a-zA-Z_:.") ];
                quoted_text = lexeme[ '"' >> *(char_ - '"') >> '"' ];

                strAttrib   = identifier >> '=' >> quoted_text >> ';';
                realAttrib  = identifier >> '=' >> double_     >> ';';
                textNode    = quoted_text;

                nodelist    = '{' >> *node >> '}';

                node        = tag
                            | strAttrib
                            | realAttrib
                            | textNode
                            ;

                tag         = '(' >> identifier >> ')' >> -nodelist;
                            ;

                // allow only tags at root of parse tree
                start       = tag;

                BOOST_SPIRIT_DEBUG_NODES((start)(tag)(node)(nodelist)(textNode)(realAttrib)(strAttrib)(quoted_text)(identifier))
            }
        };

        parserInternal::AST::Node parseSXdata2(std::string const& data) {
            typedef std::string::const_iterator It;
            typedef qi::rule<It> Skipper;

            It iter1 = data.begin();
            It iter2 = data.end();

            static const Skipper skipper = qi::space
                                         | ("/*" > *(qi::char_ - "*/") > "*/")
                                         | ("//" > *(qi::char_ - qi::eol))
                                         ;
            static const XGrammar<It, Skipper> grammar;

            parserInternal::AST::Node output;
            bool parsed = qi::phrase_parse(iter1, iter2, grammar, skipper, output);

            if(!parsed || iter1 != iter2) {
                throw std::runtime_error("Parsing failed");
            }

            return output;
        }
    }
}

int main() {
    std::cout << "AST::Node: " << sizeof(sx::parserInternal::AST::Node) << " bytes\n";
    std::cout << "XNODE_:    " << sizeof(sx::parserInternal::XNODE_)    << " bytes\n";

    std::string const data =
        "(testsx) {\n"
        "   (test) {\"hello world\"}\n"
        "   (test2) {\n"
        "       attrib1 = 123;\n"
        "       attrib2 = \"hey\";\n"
        "   }\n"
        "}";

    using namespace sx::parserInternal;
    const AST::Node output = parseSXdata2(data); //root of parsetree

    BuildLegacyXNode transform;
    XNODE_ legacy = transform(output);

    std::cout << "Root tag is named '" << legacy.name << "' and has " << legacy.nodes.size() << " direct child nodes\n";
}
Community
  • 1
  • 1
sehe
  • 374,641
  • 47
  • 450
  • 633
1

It should be

            separator %= lexeme[(char_('\n') | lit("\r\n"))];

instead of

            separator %= lexeme[(char_('\n') | char_('\r\n'))];

for starters. (Note the quotes) Looking at the rest now.

Also identifiers starting with _ are (frequently) reserved for standard library implementation - using them invoked undefined behaviour.

It seems to me that you're mixing/matching automatic attribute propagation (%=) and semantic actions rather wildly and confused. This is likely a factor in the (many) compilation issues.

In fact fixing the above issues at superficial appearance of intent, makes things compile on GCC with boost 1_55_0, as well as Clang (http://coliru.stacked-crooked.com/a/8710550143f4319a)

Still looking...

There's a largish number of things that confuse me quite a bit here.

  • Why do you fusion adapt a struct, only to use manual assignments by sequence index from tedious semnatic actions? This is labour intensive, verbose, hard to read, error prone and likely unnecessary. I mean, where you have

    strAttrib = 
        identifyer[at_c<4>(_val) = _1] >> char_('=') >> 
        textdata[at_c<5>(_val) = _1] >> char_(';')[at_c<0>(_val) = S_ATTRIB_];
    

    you could even say (without any fusion adaptation)

        strAttrib = 
            identifyer    [ phx::bind(&XNODE_::strID, _val) = _1 ]
            >> char_('=')
            >> textdata   [ phx::bind(&XNODE_::strAttrib, _val) = _1]
            >> char_(';') [ phx::bind(&XNODE_::type, _val) = S_ATTRIB_];
    

    I think you'll agree that this is better on humans and better for the compiler. And certainly better for maintenance.

  • Which brings me to the next part char_('=') and char_(';') should have been lit('=') or even just ';' there. This is also undoubtedly the reason why there is

        /**
         * bug fix - stop parser from
         * taking characters it shouldn't
         * by assigning eat with the useless
         * string sequences
         */
        std::string eat;
    

    perhaps in combination with the first observation (about %= auto-rule assignment).

  • Also, XNODE_ looks like it should really be a tagged union. If you don't want to meddle with the tricky semantics of those, you're in luck: boost has you covered with Boost Variant and, as a bonus, Spirit seemlessly integrates with that for you. So I'd naturally think of this:

        typedef boost::variant<
            Text,
            boost::recursive_wrapper<Tag>,
            SAttr,
            RAttr
        > Node;
    

    This shrinks your node type (no longer a need to have the vector footprint in each node object!), removes the need for explicit type code management (at_c<0>(_val) = XXXX?) and integrates nicely with Spirit's attribute propagation.

    I intend to show you how this will result in a cleaner grammar with far less code. Given enough time. I'll try in the next 20-30 minutes.

sehe
  • 374,641
  • 47
  • 450
  • 633
  • Preliminary results of code review are in :) Give me a few more minutes... :) – sehe Feb 12 '14 at 22:18
  • Okay, I've made some work of this. I've included any elements of "Good Spirit Style" I thought was relevant, and ended up with adding transformation of the AST to your _legacy_ :) XNODE_ type. Even with that added, the code shrunk by 100 lines. **[Read all about it](http://stackoverflow.com/a/21743177/85371)** – sehe Feb 13 '14 at 00:59