Flex C++ print code in generated file and add color

Question

I'm trying to insert the code that flex reads into my .tex file, this console app is supposed to take a .pascal and analyze it and then generate a .tex file but I'm not able to pass the code to the .tex file and then I need to add color to each token it reads, I need help!! Commands to compile it flex file.l, g++ lex.yy.c, ./a.out, test.pascal, pdflatex PDF.tex

%{ 
#include <iostream>   
#include <fstream>   
#include <string>   
using namespace std;   

int token_if = 0;   
%}


%%

if  ++token_if;
then|begin|end|procedure|function        {
                printf( "A keyword: %s\n", yytext );
                }
%%

int main(int argc, char *argv[]) 
{

    //read file
    FILE *fp;
    char filename[50], c;
    printf("Enter the filename: \n");
    scanf("%s",filename);
    fp = fopen(filename,"r");
    if (fp == NULL)
    {
        printf("file null");
        exit(0);
    } else 
    {
    yyin = fp;
    //start of lex
    yylex();    
    
    // counter
    //get_token(token_if);
    
    //create latex file
    ofstream myPDF("PDF.tex");

    myPDF  << " \\documentclass{article} "
            << "\\title{Scanner}"
            << "\\author{Andres}"
            << "\\date{III}"
            << " \\begin{document} "
            << "\\maketitle"
            << "\\newpage"
            << "\\section{¿Q?}"
            << " number of if's " 
            << token_if
            << " \\end{document} ";

        myPDF.close();

    }

    // printf("# of if's = %d",  token_if);   

    return 0;
}

Answer 1

1. (F)lex is design to split an input into a sequence of tokens. Every character in the input will be part of some identified token. What you do with the tokens is up to --you can just ignore them, if they don't interest you-- but (F)lex will still recognize them.

   A pattern list like the one you show obviously doesn't satisfy this requirement. By default, if you give (F)lex a non-exhaustive set of patterns, it adds a fallback pattern at the end, which matches any single character, and assigns it the default action, which is to send the character to yyout (by default, stdout). That's not usually what is wanted, and it's definitely not what you want.

   There is no (F)lex configuration option which changes the default action. All you can do is ensure that it never triggers, by giving (F)lex a set of patterns which is guaranteed to never fall through. You can tell Flex that you don't want its default pattern (using %option nodefault); if you do that and your patterns are not exhaustive, then Flex will produce a warning, and the scanner will immediately terminate if it can't match at some point in the scan.

   %option nodefault is highly recommended, precisely because the default action is so rarely desired. If you use that option, you will need to ensure that your patterns are complete; the simplest way to do that is to put your own fallback pattern at the end of your pattern list, with whatever action seems appropriate. If, as I suppose is the case here, you don't want to do anything with the token, you can use an action like { } or ;:

   .|\n        ;
   

2. Note that (F)lex doesn't make any assumptions about your input patterns. It does not, for example, assume that you want to ignore whitespace (maybe you don't), nor does it assume that you are only interested in the string if as a complete word. All of that is up to you. Your patters, as written, will happily recognise Eiffel and semifredo as instances of a Pascal conditional. That's surely not what you want, and I suspect you also don't want to recognise if when it is in a comment or a string literal. So you still have quite a bit of work to do on your patterns. (The usual solution to matching keywords only when they are complete words is to also match complete words, using a pattern like [[:alpha:]][[:alnum:]]*. That pattern goes after all the keyword patterns, so it will trigger if the token starts with a keyword (because (F)lex always chooses the longest match at each point in the input) but won't match if the token is precisely the keyword (because Flex always chooses the first of the patterns which match the same longest token.) It's worth reading at least two sections of the Flex manual: Patterns and How the input is matched.

3. If I undertand correctly what you are looking for, you want to know how to dump the input to a file while Flex is working on it. One obvious way would be to write each token to the file as you recognise it, which means adding boilerplate code to every action (even the actions which do nothing.) If what you are trying to do is colour-code each token, that's what you will have to do.

   If you just want to copy the parsed file to the output, all that boilerplate is a bit tedious, so you would probably be better off using the YY_USER_ACTION macro, which is implicitly expanded at the beginning of every rule action (even for rules with empty actions). Note that you will have to open the output file before you call yylex, and also write out whatever codes you need to precede the program text. Instead of writing out each token, you could choose to copy the file as it is being read by Flex. To do that, you'll need to define the YY_INPUT macro. (There is some example code in the linked manual section.)