Stack Overflow
Stack Overflow

Questions tagged [ragel]

Ragel finite-state-machine compiler

Ragel compiles a declarative machine description syntax into a source code file in Java, Ruby, or a bunch of C-like languages. In addition, Ragel can generate a graphviz .dot file containing a diagram of the states and interaction of the input machine.

Ragel can be used to generate general purpose Finite State Machines (FSMs), which are commonly used in embedded systems and in protocol-driven applications like telephony and internet servers.

Ragel can also be used to generate a "lexer" (or "scanner") - a piece of code that scans an input text and divides it into "tokens". This lexical analysis is traditionally the first step in interpreting or compiling computer languages:

input ->  lexer  ->  parser  ->  AST  ->  execution or code generation

Links

81 questions
11
votes
1 answer

How to parse template languages in Ragel?

I've been working on a parser for simple template language. I'm using Ragel. The requirements are modest. I'm trying to find [[tags]] that can be embedded anywhere in the input string. I'm trying to parse a simple template language, something that…
Tobias Lütke
  • 1,028
  • 1
  • 9
  • 9
10
votes
3 answers

Is ANTLR an appropriate tool to serialize/deserialize a binary data format?

I need to read and write octet streams to send over various networks to communicate with smart electric meters. There is an ANSI standard, ANSI C12.19, that describes the binary data format. While the data format is not overly complex the standard…
Dan Finucane
  • 1,547
  • 2
  • 18
  • 27
10
votes
3 answers

Parser Generators and Ragel... Making my own D Parser

I'm new to the world of compilers, and I recently heard about something called a parser generator. From what I (think) I've understood, parser generators take in a syntax file and output a source code file that can parse files with the given…
user541686
  • 205,094
  • 128
  • 528
  • 886
8
votes
3 answers

Simple Ragel Example that Balances Parentheses?

Here is a starting point for a grammar: %%{ machine xo; char = "x" | "o"; group = "(" char* ")"; main := group; }%% It handles (xxxx(oo)()xx), for example. How do I extend it to allow nested groups; e.g. (xxxx(o(x)o)()xx? I know recursion…
David J.
  • 31,569
  • 22
  • 122
  • 174
7
votes
9 answers

Which Java oriented lexer parser for simple project (ANTLR, DIY, etc)

I am working on a small text editor project and want to add basic syntax highlighting for a couple of languages (Java, XML..just to name a few). As a learning experience I wanted to add one of the popular or non popular Java lexer parser. What…
Berlin Brown
  • 11,504
  • 37
  • 135
  • 203
6
votes
2 answers

How to get Ragel EOF actions working

I'm working with Ragel to evaluate FSAs, and I want to embed a user action that runs whenever my machine finishes testing the input. I need this action to run regardless of whether or not the machine ends in an accepting state or not. I have this…
Mark
  • 1,746
  • 2
  • 18
  • 19
4
votes
2 answers

How do I write a simple Ragel tokenizer (no backtracking)?

UPDATE 2 Original question: Can I avoid using Ragel's |**| if I don't need backtracking? Updated answer: Yes, you can write a simple tokenizer with ()* if you don't need backtracking. UPDATE 1 I realized that asking about XML tokenizing was a red…
Seamus Abshere
  • 8,326
  • 4
  • 44
  • 61
4
votes
1 answer

How to properly scan for identifiers using Ragel

I'm trying to write a scanner for my C/C++/C#/Java/D-like programming language that I'm designing for personal reasons. For this task I'm using Ragel to generate my scanner. I'm having trouble understanding exactly when a lot of the operators…
Sion Sheevok
  • 4,057
  • 2
  • 21
  • 37
3
votes
5 answers

What does this piece of Ragel Code do?

%%{ machine microscript; action ClearNumber { currentNumber = 0; } action RecordDigit { uint8_t digit = (*p) - '0'; currentNumber = (currentNumber * 10) + digit; } number = ((digit @RecordDigit)+) >ClearNumber; …
Soham
  • 863
  • 2
  • 19
  • 35
3
votes
0 answers

Any parser that can handle recursive grammars in Go?

In a Node.js project I wrote a query parser in ANTLR4 (JS target). The user queries have a simplified SQL-like grammar that are then processed to full SQL on the server. The query structure can be arbitrarily nested. I am now porting this app to go.…
kliron
  • 4,383
  • 4
  • 31
  • 47
3
votes
1 answer

Ragel: avoid redundant call of "when" clause function

I'm writing Ragel machine for rather simple binary protocol, and what I present here is even more simplified version, without any error recovery whatsoever, just to demonstrate the problem I'm trying to solve. So, the message to be parsed here looks…
Dmitry Frank
  • 10,417
  • 10
  • 64
  • 114
3
votes
2 answers

Checking that Ragel matched the entire input

Are there better ways to require that Ragel consume all of the input? Here is what I'm using now: =begin %%{ machine my_lexer; # ... # extract tokens and store into `tokens` # ... }%% =end class MyLexer %% write data; def…
David J.
  • 31,569
  • 22
  • 122
  • 174
2
votes
1 answer

How to get Ragel to parse two names separated by (space* ":" space*)?

I'd like to parse the following: name:name where the names start and end with an alnum, and can contain any combination of alnum and spaces inside. They could also be blank. My rules for this are: identifier = alnum (space* alnum)*; name =…
user1002430
2
votes
2 answers

Failed to convert URL parser regular expression to Ragel

I found an URL parser regular expression at RFC 2396 and RFC 3986. ^(([^:\/?#]+):)?(\/\/([^\/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))? I converted it to Ragel: %%{ # RFC 3986 URI Generic Syntax (January 2005) machine url_parser; action pchar …
lxu4net
  • 2,846
  • 1
  • 16
  • 9
2
votes
2 answers

Is ragel not designed to be used with files?

Looking into ragel and can't figure out how to reasonably read from a file. As far as I understand, it requires a memory buffer that is not broken in the middle of a token. This is obviously quite a lot of work to implement, especially if I don't…
Mikolas
  • 121
  • 3
1
2 3 4 5 6