2

I would like to split a string into parts:

input = "part1/part2/part3/also3"

and fill the structure that consist of three std::string with these parts.

struct strings
{
    std::string a; // <- part1
    std::string b; // <- part2
    std::string c; // <- part3/also3
};

However my parser seems to merge the parts together and store it into the first std::string.

Here is the code on coliru

#include <iostream>

#include <boost/spirit/include/qi.hpp>
#include <boost/fusion/include/adapted.hpp>

namespace qi = ::boost::spirit::qi;

struct strings
{
    std::string a;
    std::string b;
    std::string c;
};

BOOST_FUSION_ADAPT_STRUCT(strings,
  (std::string, a) (std::string, b) (std::string, c))

template <typename It>
struct split_string_grammar: qi::grammar<It, strings ()>
{
    split_string_grammar (int parts)
        : split_string_grammar::base_type (split_string)
    {
        assert (parts > 0);

        using namespace qi;

        split_string = repeat (parts-1) [part > '/'] > last_part;

        part = +(~char_ ("/"));
        last_part = +char_;

        BOOST_SPIRIT_DEBUG_NODES ((split_string) (part) (last_part))
    }

private:
    qi::rule<It, strings ()> split_string;
    qi::rule<It, std::string ()> part, last_part;
};

int main ()
{
    std::string const input { "one/two/three/four" };

    auto const last  = input.end ();
    auto       first = input.begin ();

    // split into 3 parts.
    split_string_grammar<decltype (first)> split_string (3);
    strings ss;

    bool ok = qi::parse (first, last, split_string, ss);

    std::cout << "Parsed: " << ok << "\n";

    if (ok) {
        std::cout << "a:" << ss.a << "\n";
        std::cout << "b:" << ss.b << "\n";
        std::cout << "c:" << ss.c << "\n";
    }
}

The output is:

Parsed: 1
a:onetwo
b:three/four
c:

while I expected:

Parsed: 1
a:one
b:two
c:three/four

I'd like not to modify the grammar heavily and leave "repeat" statement in it, because the "real" grammar is much more complex of course and I will need to have it there. Just need to find the way to disable the concatenations. I tried

repeat (parts-1) [as_string[part] > '/']

but that does not compile.

Nikki Chumakov
  • 1,215
  • 8
  • 18

2 Answers2

2

The trouble here is specifically that qi::repeat is documented to expose a container of element-types.

Now, because the exposed attribute type of the rule (strings) is not a container-type, Spirit "knows" how to flatten the values.

Of course it's not what you wanted in this case, but usually this heuristic makes for really convenient accumulation of string values.

Fix 1: use a container attribute

You could witness the reverse fix by getting rid of the non-container (sequence) target attribute:

Live On Coliru

//#define BOOST_SPIRIT_DEBUG
#include <iostream>

#include <boost/spirit/include/qi.hpp>
#include <boost/fusion/include/adapted.hpp>

namespace qi = ::boost::spirit::qi;

using strings = std::vector<std::string>;

template <typename It>
struct split_string_grammar: qi::grammar<It, strings ()>
{
    split_string_grammar (int parts)
        : split_string_grammar::base_type (split_string)
    {
        assert (parts > 0);

        using namespace qi;

        split_string = repeat (parts-1) [part > '/'] 
                     > last_part
                     ;

        part         = +(~char_ ("/"))
                     ;

        last_part    = +char_
                     ;
                 
        BOOST_SPIRIT_DEBUG_NODES ((split_string) (part) (last_part))
    }

private:
    qi::rule<It, strings     ()> split_string;
    qi::rule<It, std::string ()> part, last_part;
};

int main ()
{
    std::string const input { "one/two/three/four" };

    auto const last  = input.end ();
    auto       first = input.begin ();

    // split into 3 parts.
    split_string_grammar<decltype (first)> split_string (3);
    strings ss;

    bool ok = qi::parse (first, last, split_string, ss);

    std::cout << "Parsed: " << ok << "\n";

    if (ok) {
        for(auto i = 0ul; i<ss.size(); ++i)
            std::cout << static_cast<char>('a'+i) << ":" << ss[i] << "\n";
    }
}

What you really wanted:

Of course you want to keep the struct/sequence adaptation (?); In this case that's really tricky because as soon as you use any kind of Kleene operator (*,%) or qi::repeat you'll have the attribute transformation rules as outlined above, ruining your mood.

Luckily, I just remembered I have a hacky solution based on the auto_ parser. Note the caveat in this older answer though:

Read empty values with boost::spirit

CAVEAT Specializing for std::string directly like this might not be the best idea (it might not always be appropriate and might interact badly with other parsers).

By default create_parser<std::string> is not defined, so you might decide this usage is good enough for your case:

Live On Coliru

#include <boost/fusion/adapted/struct.hpp>
#include <boost/spirit/include/qi.hpp>

namespace qi = boost::spirit::qi;

struct strings {
    std::string a;
    std::string b;
    std::string c;
};

namespace boost { namespace spirit { namespace traits {
    template <> struct create_parser<std::string> {
        typedef proto::result_of::deep_copy<
            BOOST_TYPEOF(
                qi::lexeme [+(qi::char_ - '/')] | qi::attr("(unspecified)")
            )
        >::type type;

        static type call() {
            return proto::deep_copy(
                qi::lexeme [+(qi::char_ - '/')] | qi::attr("(unspecified)")
            );
        }
    };
}}}

BOOST_FUSION_ADAPT_STRUCT(strings, (std::string, a)(std::string, b)(std::string, c))

template <typename Iterator>
struct google_parser : qi::grammar<Iterator, strings()> {
    google_parser() : google_parser::base_type(entry, "contacts") {
        using namespace qi;

        entry =
                skip('/') [auto_]
              ;
    }
  private:
    qi::rule<Iterator, strings()> entry;
};

int main() {
    using It = std::string::const_iterator;
    google_parser<It> p;

    std::string const input = "part1/part2/part3/also3";
    It f = input.begin(), l = input.end();

    strings ss;
    bool ok = qi::parse(f, l, p >> *qi::char_, ss, ss.c);

    if (ok)
    {
        std::cout << "a:" << ss.a << "\n";
        std::cout << "b:" << ss.b << "\n";
        std::cout << "c:" << ss.c << "\n";
    }
    else
        std::cout << "Parse failed\n";

    if (f!=l)
        std::cout << "Remaining unparsed: '" << std::string(f,l) << "'\n";
}

Prints

a:part1
b:part2
c:part3/also3

Update/Bonus

In reponse to the OP's own answer I wanted to challenge myself to write it more generically indeed.

The main thing is to to write set_field_ in such a way that it doesn't know/assume more than required about the destination sequence type.

With a bit of Boost Fusion magic that became:

struct set_field_
{
    template <typename Seq, typename Value>
    void operator() (Seq& seq, Value const& src, unsigned idx) const {
        fus::fold(seq, 0u, Visit<Value> { idx, src });
    }
private:
    template <typename Value>
    struct Visit {
        unsigned     target_idx;
        Value const& value;

        template <typename B>
        unsigned operator()(unsigned i, B& dest) const {
            if (target_idx == i) {
                boost::spirit::traits::assign_to(value, dest);
            }
            return i + 1;
        }
    };
};

It has the added flexibility of applying Spirit's attribute compatibility rules¹. So, you can use the same grammar with both the following types:

struct strings {
    std::string a, b, c;
};

struct alternative {
    std::vector<char> first;
    std::string       second;
    std::string       third;
};

To drive the point home, I made the adaptation of the second struct reverse the field order:

BOOST_FUSION_ADAPT_STRUCT(strings, a, b, c)
BOOST_FUSION_ADAPT_STRUCT(alternative, third, second, first) // REVERSE ORDER :)

Without further ado, the demo program:

Live On Coliru

#define BOOST_SPIRIT_USE_PHOENIX_V3
#define BOOST_RESULT_OF_USE_DECLTYPE
#include <boost/fusion/adapted.hpp>
#include <boost/fusion/algorithm/iteration.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>

namespace qi  = boost::spirit::qi;
namespace fus = boost::fusion;

struct strings {
    std::string a, b, c;
};

struct alternative {
    std::vector<char> first;
    std::string       second;
    std::string       third;
};

BOOST_FUSION_ADAPT_STRUCT(strings, a, b, c)
BOOST_FUSION_ADAPT_STRUCT(alternative, third, second, first) // REVERSE ORDER :)

// output helpers for demo:
namespace {
    inline std::ostream& operator<<(std::ostream& os, strings const& data) {
        return os 
            << "a:\"" << data.a << "\" " 
            << "b:\"" << data.b << "\" " 
            << "c:\"" << data.c << "\" ";
    }

    inline std::ostream& operator<<(std::ostream& os, alternative const& data) {
        os << "first: vector<char> { \""; os.write(&data.first[0], data.first.size()); os << "\" } ";
        os << "second: \"" << data.second << "\" ";
        os << "third: \""  << data.third  << "\" ";
        return os;
    }
}

struct set_field_
{
    template <typename Seq, typename Value>
    void operator() (Seq& seq, Value const& src, unsigned idx) const {
        fus::fold(seq, 0u, Visit<Value> { idx, src });
    }
  private:
    template <typename Value>
    struct Visit {
        unsigned     target_idx;
        Value const& value;

        template <typename B>
        unsigned operator()(unsigned i, B& dest) const {
            if (target_idx == i) {
                boost::spirit::traits::assign_to(value, dest);
            }
            return i + 1;
        }
    };
};

boost::phoenix::function<set_field_> const set_field = {};

template <typename It, typename Target>
struct split_string_grammar: qi::grammar<It, Target(), qi::locals<unsigned> >
{
    split_string_grammar (int parts)
        : split_string_grammar::base_type (split_string)
    {
        assert (parts > 0);

        using namespace qi;
        using boost::phoenix::val;

        _a_type _current; // custom placeholder

        split_string = 
              eps       [ _current = 0u ]
            > repeat (parts-1) 
                [part   [ set_field(_val, _1, _current++) ] > '/']
            > last_part [ set_field(_val, _1, _current++) ];

        part = +(~char_ ("/"));
        last_part = +char_;

        BOOST_SPIRIT_DEBUG_NODES ((split_string) (part) (last_part))
    }

private:
    qi::rule<It, Target(), qi::locals<unsigned> > split_string;
    qi::rule<It, std::string()> part, last_part;
};

template <size_t N = 3, typename Target>
void run_test(Target target) {
    using It = std::string::const_iterator;
    std::string const input { "one/two/three/four" };

    It first = input.begin(), last = input.end();

    split_string_grammar<It, Target> split_string(N);

    bool ok = qi::parse (first, last, split_string, target);

    if (ok) {
        std::cout << target << '\n';
    } else {
        std::cout << "Parse failed\n";
    }

    if (first != last)
        std::cout << "Remaining input left unparsed: '" << std::string(first, last) << "'\n";
}

int main ()
{
    run_test(strings {});
    run_test(alternative {});
}

Output:

a:"one" b:"two" c:"three/four" 
first: vector<char> { "three/four" } second: "two" third: "one" 

¹ as with BOOST_SPIRIT_ACTIONS_ALLOW_ATTR_COMPAT

Community
  • 1
  • 1
sehe
  • 374,641
  • 47
  • 450
  • 633
  • @NikkiChumakov Thank you :) I was most pleased with the `traits::assign_to` touch I found there. I learned that. – sehe Jul 22 '15 at 19:56
2

Besides sehe's suggestions one more possible way is to use semantic actions (coliru):

struct set_field_
{
    void operator() (strings& dst, std::string const& src, unsigned& idx) const
    {
        assert (idx < 3);
        switch (idx++) {
            case 0: dst.a = src; break;
            case 1: dst.b = src; break;
            case 2: dst.c = src; break;
        }
    }
};

boost::phoenix::function<set_field_> const set_field { set_field_ {} };

template <typename It>
struct split_string_grammar: qi::grammar<It, strings (), qi::locals<unsigned> >
{
    split_string_grammar (int parts)
        : split_string_grammar::base_type (split_string)
    {
        assert (parts > 0);

        using namespace qi;
        using boost::phoenix::val;

        split_string = eps [ _a = val (0) ]
            > repeat (parts-1) [part [ set_field (_val, _1, _a) ] > '/']
            > last_part [ set_field (_val, _1, _a) ];

        part = +(~char_ ("/"));
        last_part = +char_;

        BOOST_SPIRIT_DEBUG_NODES ((split_string) (part) (last_part))
    }

private:
    qi::rule<It, strings (), qi::locals<unsigned> > split_string;
    qi::rule<It, std::string ()> part, last_part;
};
Community
  • 1
  • 1
Nikki Chumakov
  • 1,215
  • 8
  • 18
  • I thought of this one too (and then dismissed it). All of these examples leave me thinking you're better off just using a split algorithm :) (PS. Use `fusion::at_v(_val)` so you can avoid the external functions) – sehe Jul 21 '15 at 06:26
  • @sehe First of all, thank you for your excellent quality answers. Without it I will never be able even to understand why the rule does not work as I expected. I choose solution with actions only because it is not "invasive" - does not modify the behavior of the parser in other cases and classes. That is the only reason. PS. I don't get an idea how can I get rid of external action, because I auto-increment the 'part number' (_a) value in my action. I'd like to avoid custom actions, but do not know how I can enumerate parts from 0 to last-part properly with pure fusion or phoenix. – Nikki Chumakov Jul 21 '15 at 17:45
  • You're right. That was a thinko. The types of `at_c` are different :( Of course, you can still make it work generically for a fusion (adapted) sequence. So you don't have to duplicate the class layout there – sehe Jul 21 '15 at 19:51
  • For fun and glory, I added the answer with the generic approach to my answer. Demo [Live On Coliru](http://coliru.stacked-crooked.com/a/8d3719bed5ec2279) – sehe Jul 21 '15 at 22:05