rule to extract key+phrases from a text document

Question

I want to extract the key phrases from the document: "something KEY phrase END something ... ect". My rule works well but the result does not contain of key name. What should be the rule in order to get a string: "KEY phrase". Thank you for the advice.

std::vector<std::string> doc; 
bool r = qi::phrase_parse(first,last, 
  ( qi::omit[*(qi::char_-"KEY")] 
    >> qi::lexeme[ "KEY"
    >> *(qi::char_-"KEY" -"END")] ) % "END"
, qi::space, doc);

(Also, you forgot to tell us the type of `doc`) – sehe Jan 04 '15 at 19:40 — sehe, Jan 04 '15 at 19:40

sehe · Accepted Answer · 2015-01-04T20:19:05.460

qi::lit(...) doesn't synthesize an attribute.

qi::string(...) does.

Replace "KEY" with qi::string("KEY"), likely. (hard to tell without knowing the type of doc)

bool r = qi::phrase_parse(first,last, 
  ( qi::omit[*(qi::char_-"KEY")] 
    >> qi::lexeme[ qi::string("KEY")
    >> *(qi::char_-"KEY" -"END")] ) % "END"
, qi::space, doc);

BONUS See also seek[] parser directive from the Spirit Repository:

The seek[] parser-directive skips all input until the subject parser matches.

Here's what I'd do:

Live On Coliru

#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/repository/include/qi_seek.hpp>
namespace qi = boost::spirit::qi;
namespace qr = boost::spirit::repository::qi;

extern std::string const sample; // below

int main() {
    auto f(sample.begin()), l(sample.end());

    std::vector<std::string> phrases;

    if (qi::parse(f,l, *qi::as_string[
                qr::seek[qi::string("KEY")] >> *(qi::char_ - "END")
            ], phrases)) 
    {
        for (size_t i = 0; i < phrases.size(); ++i) 
            std::cout << "keyphrase #" << i << ": '" << phrases[i] << "'\n";
    }
}

Prints:

keyphrase #0: 'KEY@v/0qwJTjgFQwNmose7LiEmAmKpIdK3TPmkCs@'
keyphrase #1: 'KEY@G1TErN1QSSKi17BSnwBKML@'
keyphrase #2: 'KEY@pWhBKmc0sD+o@'
keyphrase #3: 'KEY@pwgjNJ0FvWGRezwi74QdIQdmUuKVyquWuvXz4tBOXqMMqco@'
keyphrase #4: 'KEY@aJ3QUfLh3AqfKyxcUSiDbanZmCNGza6jb6pZ@'
keyphrase #5: 'KEY@bYJzitZUyXlgPA009qBpleHIJ9uJUSdJO78iisUgHkoqUpf+oXZQF9X/7v2fikgemCD@'

Sample data included in a comment in this answer: /here/

@stansy I've included a sample, based on `repository::qi::seek` — sehe, Jan 04 '15 at 19:38
Welcome to SO :) See also [how does accepting an answer work](http://meta.stackoverflow.com/questions/5234/how-does-accepting-an-answer-work). Cheers — sehe, Jan 04 '15 at 20:07

stansy · Answer 2 · 2015-01-05T16:42:00.783

0

Final benchmark:

Spirit Qi (sahe's solution above) - Items count: 5585 Elapsed time: 183.676 seconds
Boost regex - Items count: 5585 Elapsed time: 197.684 seconds
Xpressive regex - Items count: 5585 Elapsed time: 232.597 seconds

Data size: 23.9 MB text file

edited Jan 05 '15 at 16:42

answered Jan 05 '15 at 16:29

stansy

135
1
10

rule to extract key+phrases from a text document

2 Answers2

Linked