Regular expressions in yacc's code with PLY

Question

I am writing regular expressions with PLY but it seems it doesn't behave like yacc.

I've written this:

def p_expression_TYPE (p):
'''
    TYPE :  [' Discrete ',  ' Continuous ' ]
'''

to express the fact that a TYPE can only take those values: Discrete, or Continuous.

I have the followed errors:

ERROR: la_grammaire_yacc.py:31: Illegal name '['' in rule 'TYPE'

Could somebody tell me what's wrong with my code? Must I definethose king of expressions as tokens?

Thanks to you

Answer 1

Regular expressions and yacc are different kinds of animals. It helps to understand the difference between Chomsky Type 3 and Chomsky type 2 languages. Regular expression, which are used by flex and the lexical analysis part of PLY are Chomsky type 3. Yacc, and the grammar rules part of PLY the representation is for a Context Free Grammar, which is Chomsky type 2.

They have different purposes, different characteristics and different notations. As a result you cannot take a short-hand, or rule from one to the other. However, everything you can write in a regular expression can be handled by a context free grammar; just written differently. (Just a bit of computer science theory creeping in there).

It's also worth consulting a good manual on PLY, which has similar examples.

So, for your example, the syntax rule for TYPE might be written:

def p_expression_TYPE (p):
'''
    TYPE : 'Discrete'
         | 'Continuous' 
'''

Unfortunately, PLY has a restriction that any character literals, such as 'Discrete' and 'Continuous' must be declared as tokens in the lex part:

import ply.lex as lex

# List of token names.   This is always required
tokens = (
   'DISCRETE',
   'CONTINUOUS'
)

# Regular expression rules for simple tokens
t_DISCRETE    = r'Discrete'
t_CONTINUOUS   = r'Continuous' 

Now we can put the parser in yacc.py as

def p_expression_TYPE (p):
'''
    TYPE : DISCRETE
         | CONTINUOUS 
'''