Unsure about error Building Flex and Bison Compiler from Link

Question

I was trying to compiler from http://gnuu.org/2009/09/18/writing-your-own-toy-compiler and have been having some issues with it. https://github.com/lsegal/my_toy_compiler

When using the make file that comes with it Get the error

C:\New Compile\my_toy_compiler-master\my_toy_compiler-master>make
bison -d -o parser.cpp parser.y
parser.y: warning: 48 shift/reduce conflicts [-Wconflicts-sr]
g++ -c `llvm-config --cppflags` -std=c++11 -o parser.o parser.cpp
g++ -c `llvm-config --cppflags` -std=c++11 -o codegen.o codegen.cpp
g++ -c `llvm-config --cppflags` -std=c++11 -o main.o main.cpp
flex -o tokens.cpp tokens.l parser.hpp
g++ -c `llvm-config --cppflags` -std=c++11 -o tokens.o tokens.cpp
tokens.cpp: In function 'void yy_init_buffer(YY_BUFFER_STATE, FILE*)':
tokens.cpp:1464:48: error: 'fileno' was not declared in this scope

                                                ^
tokens.cpp:1464:48: note: suggested alternative: 'file'

                                                ^
                                                file
make: *** [Makefile:27: tokens.o] Error 1

I looked at the file generated by the lexer and copied the method,

    static void yy_init_buffer  (YY_BUFFER_STATE  b, FILE * file )

{
    int oerrno = errno;

    yy_flush_buffer( b );

    b->yy_input_file = file;
    b->yy_fill_buffer = 1;

    /* If b is the current buffer, then yy_init_buffer was _probably_
     * called from yyrestart() or through yy_get_next_buffer.
     * In that case, we don't want to reset the lineno or column.
     */
    if (b != YY_CURRENT_BUFFER){
        b->yy_bs_lineno = 1;
        b->yy_bs_column = 0;
    }

        b->yy_is_interactive = file ? (isatty( fileno(file) ) > 0) : 0;

    errno = oerrno;
}

And using the regular way given in the blog I get a numerous amount of errors.

C:\New Compile\my_toy_compiler-master\my_toy_compiler-master>g++ -o parser parser.cpp tokens.cpp main.cpp
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc6vUDwi.o:parser.cpp:(.rdata$.refptr._ZTV20NFunctionDeclaration[.refptr._ZTV20NFunctionDeclaration]+0x0): undefined reference to `vtable for NFunctionDeclaration'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc6vUDwi.o:parser.cpp:(.rdata$.refptr._ZTV18NExternDeclaration[.refptr._ZTV18NExternDeclaration]+0x0): undefined reference to `vtable for NExternDeclaration'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc6vUDwi.o:parser.cpp:(.rdata$.refptr._ZTV20NVariableDeclaration[.refptr._ZTV20NVariableDeclaration]+0x0): undefined reference to `vtable for NVariableDeclaration'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc6vUDwi.o:parser.cpp:(.rdata$.refptr._ZTV16NReturnStatement[.refptr._ZTV16NReturnStatement]+0x0): undefined reference to `vtable for NReturnStatement'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc6vUDwi.o:parser.cpp:(.rdata$.refptr._ZTV20NExpressionStatement[.refptr._ZTV20NExpressionStatement]+0x0): undefined reference to `vtable for NExpressionStatement'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc6vUDwi.o:parser.cpp:(.rdata$.refptr._ZTV6NBlock[.refptr._ZTV6NBlock]+0x0): undefined reference to `vtable for NBlock'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc6vUDwi.o:parser.cpp:(.rdata$.refptr._ZTV11NAssignment[.refptr._ZTV11NAssignment]+0x0): undefined reference to `vtable for NAssignment'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc6vUDwi.o:parser.cpp:(.rdata$.refptr._ZTV15NBinaryOperator[.refptr._ZTV15NBinaryOperator]+0x0): undefined reference to `vtable for NBinaryOperator'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc6vUDwi.o:parser.cpp:(.rdata$.refptr._ZTV11NMethodCall[.refptr._ZTV11NMethodCall]+0x0): undefined reference to `vtable for NMethodCall'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc6vUDwi.o:parser.cpp:(.rdata$.refptr._ZTV11NIdentifier[.refptr._ZTV11NIdentifier]+0x0): undefined reference to `vtable for NIdentifier'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc6vUDwi.o:parser.cpp:(.rdata$.refptr._ZTV7NDouble[.refptr._ZTV7NDouble]+0x0): undefined reference to `vtable for NDouble'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc6vUDwi.o:parser.cpp:(.rdata$.refptr._ZTV8NInteger[.refptr._ZTV8NInteger]+0x0): undefined reference to `vtable for NInteger'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text+0x24): undefined reference to `LLVMLinkInMCJIT'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text+0x24): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `LLVMLinkInMCJIT'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text+0x9e): undefined reference to `createCoreFunctions(CodeGenContext&)'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text+0x9e): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `createCoreFunctions(CodeGenContext&)'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text+0xb4): undefined reference to `CodeGenContext::generateCode(NBlock&)'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text+0xb4): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `CodeGenContext::generateCode(NBlock&)'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text+0xc3): undefined reference to `CodeGenContext::runCode()'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text+0xc3): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `CodeGenContext::runCode()'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text+0x16e): undefined reference to `llvm::LLVMContext::LLVMContext()'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text+0x16e): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `llvm::LLVMContext::LLVMContext()'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text$_ZN4llvm22InitializeNativeTargetEv[_ZN4llvm22InitializeNativeTargetEv]+0x9): undefined reference to `LLVMInitializeX86TargetInfo'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text$_ZN4llvm22InitializeNativeTargetEv[_ZN4llvm22InitializeNativeTargetEv]+0x9): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `LLVMInitializeX86TargetInfo'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text$_ZN4llvm22InitializeNativeTargetEv[_ZN4llvm22InitializeNativeTargetEv]+0xe): undefined reference to `LLVMInitializeX86Target'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text$_ZN4llvm22InitializeNativeTargetEv[_ZN4llvm22InitializeNativeTargetEv]+0xe): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `LLVMInitializeX86Target'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text$_ZN4llvm22InitializeNativeTargetEv[_ZN4llvm22InitializeNativeTargetEv]+0x13): undefined reference to `LLVMInitializeX86TargetMC'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text$_ZN4llvm22InitializeNativeTargetEv[_ZN4llvm22InitializeNativeTargetEv]+0x13): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `LLVMInitializeX86TargetMC'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text$_ZN4llvm32InitializeNativeTargetAsmPrinterEv[_ZN4llvm32InitializeNativeTargetAsmPrinterEv]+0x9): undefined reference to `LLVMInitializeX86AsmPrinter'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text$_ZN4llvm32InitializeNativeTargetAsmPrinterEv[_ZN4llvm32InitializeNativeTargetAsmPrinterEv]+0x9): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `LLVMInitializeX86AsmPrinter'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text$_ZN4llvm31InitializeNativeTargetAsmParserEv[_ZN4llvm31InitializeNativeTargetAsmParserEv]+0x9): undefined reference to `LLVMInitializeX86AsmParser'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text$_ZN4llvm31InitializeNativeTargetAsmParserEv[_ZN4llvm31InitializeNativeTargetAsmParserEv]+0x9): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `LLVMInitializeX86AsmParser'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text$_ZN14CodeGenContextC1Ev[_ZN14CodeGenContextC1Ev]+0x73): undefined reference to `llvm::Module::Module(llvm::StringRef, llvm::LLVMContext&)'
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.text$_ZN14CodeGenContextC1Ev[_ZN14CodeGenContextC1Ev]+0x73): additional relocation overflows omitted from the output
/cygdrive/c/Users/Joey/AppData/Local/Temp/cc4uUJR7.o:main.cpp:(.rdata$.refptr._ZN4llvm11LLVMContextD1Ev[.refptr._ZN4llvm11LLVMContextD1Ev]+0x0): undefined reference to `llvm::LLVMContext::~LLVMContext()'
collect2: error: ld returned 1 exit status

C:\New Compile\my_toy_compiler-master\my_toy_compiler-master>g++ -o parser `llvm-config --libs core jit native --cxxflags --ldflags` *.cpp
g++: error: `llvm-config: No such file or directory
g++: error: core: No such file or directory
g++: error: jit: No such file or directory
g++: error: native: No such file or directory
g++: error: unrecognized command line option '--libs'
g++: error: unrecognized command line option '--cxxflags'
g++: error: unrecognized command line option '--ldflags`'

C:\New Compile\my_toy_compiler-master\my_toy_compiler-master>

I'm unsure what step to take next, as I played around with Cygwin and the different packages. At first I thought perhaps it was the version as the Github states it only supports llvm up to 4.0, so I backtracked and installed 4.0 instead, and still came with the same issue.

Answer 1

Flex can generate three kinds of lexers:

• Interactive

• Non-interactive ("batch")

• Sometimes interactive (the default).

Non-interactive lexers are the most efficient; they read input a buffer at a time. That's great for parsing files, but it's annoying for parsing console input, particularly when the intent is to respond to each line typed. Interactive lexers avoid this problem by reading input one character at a time, so that a token is reported as soon as possible. But that can be inefficient for input with large tokens.

The default lexer tests to see if its input is a console and uses that information to choose between interactive and non-interactive buffer handling. The only problem with this strategy is that the standard C library does not provide any way to tell if an input stream is a console. Unix(-like) systems do have a function which answers that question, but it doesn't work with standard C FILE* streams; it requires the underlying "file number". The fileno function extracts the underlying file number from a FILE*, but it is obviously not a standard C function.

So if you have a default lexer, it needs to call fileno. However, fileno is not declared in any standard header unless you #define a feature-test macro. But you have to define these macros right at the beginning of the C program, which is hard to do when the C program is generated automatically. The easiest solution is to add the macro definition on the compiler command line using the -D option:

g++ -c `llvm-config --cppflags` -D_POSIX_C_SOURCE -std=c++11 -o tokens.o tokens.cpp

This is just one of the possible macros which you could use; I chose it here because it is the one in man fileno, but I usually use -D_XOPEN_SOURCE=700, which requests interfaces for a more recent Posix version. Another possibility is _GNU_SOURCE, which gets the Gnu headers to pretty well include everything, including Gnu-specific extension functions.

On my (linux) system, llvm-config --cppflags includes -D_GNU_SOURCE, but that doesn't really matter because current versions of both g++ and clang++ automatically define _GNU_SOURCE for C++ targets. Apparently this is not the case with Cygwin. See, for example:

• GoogleTest 1.6 with Cygwin 1.7 compile error: 'fileno' was not declared in this scope).

• error: ‘fileno’ was not declared in this scope

both of which suggest changing -std=c++11 to -std=gnu++11.

An alternative is to side-step the issue by requesting a non-interactive lexer, either with a flex command-line option:

flex --batch -o tokens.cpp tokens.l parser.hpp

Or by adding

%option batch

to your flex input file. See the flex manual

(Obviously, don't request a batch lexer if you are writing an interactive calculator. In that case, you would probably want to request an always-interactive lexer.)