CMake FindFLEX produces NOTFOUND on windows

Question

I installed flex and bison with chocolatey

choco install winflexbison3

and created this CMakeLists.txt

find_package(BISON)
find_package(FLEX)
message("FLEX_FOUND: ${FLEX_FOUND}")
message("FLEX_EXECUTABLE: ${FLEX_EXECUTABLE}")
message("FLEX_INCLUDE_DIRS: ${FLEX_INCLUDE_DIRS}")
message("FLEX_LIBRARIES: ${FLEX_LIBRARIES}")

BISON_TARGET(MyParser parser.y ${CMAKE_CURRENT_BINARY_DIR}/parser.cpp)
FLEX_TARGET(MyScanner lexer.l  ${CMAKE_CURRENT_BINARY_DIR}/lexer.cpp)
ADD_FLEX_BISON_DEPENDENCY(MyScanner MyParser)
include_directories(${CMAKE_CURRENT_BINARY_DIR})
 add_executable(Foo
    ${BISON_MyParser_OUTPUTS}
    ${FLEX_MyScanner_OUTPUTS}
 )
 target_link_libraries(Foo ${FLEX_LIBRARIES})

I wrote a dummy parser (copied from https://aquamentus.com/flex_bison.html) and tried to build the project

-- Found BISON: C:/ProgramData/chocolatey/bin/win_bison.exe (found version "3.7.4")
-- Found FLEX: C:/ProgramData/chocolatey/bin/win_flex.exe (found version "2.6.4")
FLEX_FOUND: TRUE
FLEX_EXECUTABLE: C:/ProgramData/chocolatey/bin/win_flex.exe
FLEX_INCLUDE_DIRS: FLEX_INCLUDE_DIR-NOTFOUND
FLEX_LIBRARIES: FL_LIBRARY-NOTFOUND
CMake Warning (dev) in CMakeLists.txt:
  No cmake_minimum_required command is present.  A line of code such as

    cmake_minimum_required(VERSION 3.23)

  should be added at the top of the file.  The version specified may be lower
  if you wish to support older CMake versions for this project.  For more
  information run "cmake --help-policy CMP0000".
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring done
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
FL_LIBRARY (ADVANCED)
    linked by target "Foo" in directory C:/Users/Aleksander/source/repos/C/insilico/blender/test

-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

For some reason CMake can't find the library and include dir. What should I do? Where does FLEX expect the libraries to be?

This is what I did so far:

I see FlexLexer.h to be right there

C:\ProgramData\chocolatey\lib\winflexbison3\tools> ls
FlexLexer.h          changelog.md           custom_build_rules  win_bison.exe
UNISTD_ERROR.readme  chocolateyInstall.ps1  data                win_flex.exe

I see that FindFLEX uses this source code to look for this file but somehow fails https://github.com/Kitware/CMake/blob/241fc839d56ccd666fe41269e291b8d8190cf97b/Modules/FindFLEX.cmake#L117

find_library(FL_LIBRARY NAMES fl
  DOC "Path to the fl library")

find_path(FLEX_INCLUDE_DIR FlexLexer.h
  DOC "Path to the flex headers")

mark_as_advanced(FL_LIBRARY FLEX_INCLUDE_DIR)

set(FLEX_INCLUDE_DIRS ${FLEX_INCLUDE_DIR})
set(FLEX_LIBRARIES ${FL_LIBRARY})

I found this documentation for find_path http://devdoc.net/linux/cmake-3.9.6/command/find_path.html and there is a list of paths that are searched. I queried them with

message("CMAKE_INCLUDE_PATH: ${CMAKE_INCLUDE_PATH}")
message("CMAKE_FRAMEWORK_PATH: ${CMAKE_FRAMEWORK_PATH}")
message("CMAKE_SYSTEM_INCLUDE_PATH: ${CMAKE_SYSTEM_INCLUDE_PATH}")
message("CMAKE_SYSTEM_FRAMEWORK_PATH: ${CMAKE_SYSTEM_FRAMEWORK_PATH}")
message("CMAKE_LIBRARY_ARCHITECTURE: ${CMAKE_LIBRARY_ARCHITECTURE}")
message("CMAKE_FIND_ROOT_PATH: ${CMAKE_FIND_ROOT_PATH}")
message("CMAKE_SYSROOT: ${CMAKE_SYSROOT}")

and got

CMAKE_INCLUDE_PATH:
CMAKE_FRAMEWORK_PATH:
CMAKE_SYSTEM_INCLUDE_PATH:
CMAKE_SYSTEM_FRAMEWORK_PATH:
CMAKE_LIBRARY_ARCHITECTURE:
CMAKE_FIND_ROOT_PATH:
CMAKE_SYSROOT:

So it seems that find_path does absolutely nothing. Should I specify the paths myself? But is such case what is CMake even for?

Answer 1

It's astounding how many irritating problems are caused by a trivial library file which probably shouldn't exist anyway, and which is hardly ever needed.

The library in question was originally called libl (l for lex), and Posix requires that when you link executables which include a lex-generated scanner, you add -ll to the linker invocation. Flex, which to all intents and purposes is the current successor to the Lex tool, decided to change the name of the library to libfl, and that's generally what you'll find in a Flex installation. So the usual instructions you'll find these days (i.e., for the last several decades) say that you should use the command-line option -lfl.

Often, but not always, distributions add a symbolic link (or equivalent) for libl so that the Posix-specified -ll still works. Occasionally, the Flex distribution is modified so that the library is called libl, in which case -lfl won't work. And sometimes, the library isn't part of the distribution at all and needs to be acquired from a different package.

Posix wanted to cater for a Lex implementation which puts some functions into a library instead of generating the same code each time. But few (if any) lex implementations ever took advantage of that provision; in particular, Flex-generated scanners have no reliance on libfl for any internal function. There are only two functions defined in that library:

int main(void) { while (yylex()) {} }

int yywrap(void) { return 1; }

The first one is a default implementation of main which just calls yylex until it returns 0 (indicating EOF). That can be handy for quick-and-dirty scanner rule tests, but any real project will certainly have a main() definition.

The second provides a default yywrap implementation. However, Flex provides an even simpler way of indicating that the yywrap functionality isn't required:

%option noyywrap

If you don't require yywrap, you should just add the above to your .l file (in the prologue). If you do require yywrap, then you need to define it. In either case, you don't need the library.

So my recommendation is that rather wasting your time trying to figure out the deficiencies of the flex packaging you're using, just add the above %option to your Flex input file and eliminate the library from your build rules.