Simple parser and generator

Question

I need to parse and generate some texts from and to c++ objects.

The syntax is:

command #param #param #param

There is set of commands some of them have no params etc. Params are mainly numbers.

The question is: Should I use Boost Spirit for this task? Or just simply tokenize each line evaluate function to call from string compare with command, read additional parameters and create cpp object from it?

If you suggest using Spirit or any other solution it would be nice if you could provide some examples similiar to my problem. I've read and tried all examples from Boost Spirit doc.

If you want to learn Spirit, go ahead. Otherwise this is definitely simple enough that I would just tokenize manually. — tenfour, Jul 09 '12 at 14:02
Take a look to boost::wave. Here is example http://stackoverflow.com/questions/2666310/is-there-a-tokenizer-for-a-cpp-file — mirt, Jul 09 '12 at 14:11

score 5 · Accepted Answer · edited May 23 '17 at 12:27

I implemented more or less precisely this in a previous answer to the question " Using boost::bind with boost::function: retrieve binded variable type ".

The complete working sample program (which expects a very similar grammar) using Boost Spirit is here: https://gist.github.com/1314900. You'd just want to remove the /execute literals for your grammar, so edit Line 41 from

    if (!phrase_parse(f,l, "/execute" > (

to

    if (!phrase_parse(f,l, (

The example script

WriteLine "bogus"
Write     "here comes the answer: "
Write     42
Write     31415e-4
Write     "that is the inverse of" 24 "and answers nothing"
Shutdown  "Bye" 9
Shutdown  "Test default value for retval"

Now results in the following output after execution:

WriteLine('bogus');
Write(string: 'here comes the answer: ');
Write(double: 42);
Write(double: 3.1415);
Write(string: 'that is the inverse of');
Write(double: 24);
Write(string: 'and answers nothing');
Shutdown(reason: 'Bye', retval: 9)
Shutdown(reason: 'Test default value for retval', retval: 0)

Full Code

For archival purposes:

#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <fstream>

namespace qi  = boost::spirit::qi;
namespace phx = boost::phoenix;

///////////////////////////////////
// 'domain classes' (scriptables)
struct Echo
{
    void WriteLine(const std::string& s) { std::cout << "WriteLine('"     << s << "');" << std::endl; }
    void WriteStr (const std::string& s) { std::cout << "Write(string: '" << s << "');" << std::endl; }
    void WriteInt (int i)                { std::cout << "Write(int: "     << i <<  ");" << std::endl; }
    void WriteDbl (double d)             { std::cout << "Write(double: "  << d <<  ");" << std::endl; }
    void NewLine  ()                     { std::cout << "NewLine();"                    << std::endl; }
} echoService;

struct Admin
{
    void Shutdown(const std::string& reason, int retval) 
    { 
        std::cout << "Shutdown(reason: '" << reason << "', retval: " << retval << ")" << std::endl;
        // exit(retval);
    }
} adminService;

void execute(const std::string& command)
{
    typedef std::string::const_iterator It;
    It f(command.begin()), l(command.end());

    using namespace qi;
    using phx::bind;
    using phx::ref;

    rule<It, std::string(), space_type> stringlit = lexeme[ '"' >> *~char_('"') >> '"' ];

    try
    {
        if (!phrase_parse(f,l, /*"/execute" >*/ (
                (lit("WriteLine")  
                    > stringlit  [ bind(&Echo::WriteLine, ref(echoService), _1) ])
              | (lit("Write") >> +(
                      double_    [ bind(&Echo::WriteDbl,  ref(echoService), _1) ] // the order matters
                    | int_       [ bind(&Echo::WriteInt,  ref(echoService), _1) ]
                    | stringlit  [ bind(&Echo::WriteStr,  ref(echoService), _1) ]
                ))
              | (lit("NewLine")  [ bind(&Echo::NewLine,   ref(echoService)) ])
              | (lit("Shutdown")  > (stringlit > (int_ | attr(0))) 
                                 [ bind(&Admin::Shutdown, ref(adminService), _1, _2) ])
            ), space))
        {
            if (f!=l) // allow whitespace only lines
                std::cerr << "** (error interpreting command: " << command << ")" << std::endl;
        }
    }
    catch (const expectation_failure<It>& e)
    {
        std::cerr << "** (unexpected input '" << std::string(e.first, std::min(e.first+10, e.last)) << "') " << std::endl;
    }

    if (f!=l)
        std::cerr << "** (warning: skipping unhandled input '" << std::string(f,l) << "')" << std::endl;
}

int main()
{
    std::ifstream ifs("input.txt");

    std::string command;
    while (std::getline(ifs/*std::cin*/, command))
        execute(command);
}

score 1 · Answer 2 · answered Jul 09 '12 at 14:39

For simple formatted, easily tested input, tokenizing should be enough.
When tokenizing, you can read a line from the input and put that in a stringstream (iss). From iss, you read the first word and pass that to a command factory which creates the right command for you. Then you can pass iss to the readInParameters function of the new command, so each command can parse it own parameters and check whether all parameters are valid.

Not tested code-sample:

std::string line;
std::getline(inputStream, line);
std::istringstream iss(line);
std::string strCmd;
iss >> strCmd;
try
{
  std::unique_ptr<Cmd> newCmd = myCmdFactory(strCmd);
  newCmd->readParameters(iss);
  newCmd->execute();
  //...
}
catch (std::exception& e) 
{ 
  std::cout << "Issue with received command: " << e.what() << "\n"; 
}

Simple parser and generator

2 Answers2

Full Code