How can make CMake install an external project my targets depend on?

Question

I am developing an application and have separated some functionality into a second repository. I want to

1. Make the library repo a git submodule since both will be developed side-by-side
2. Include the library as a dependency in my CMakeLists without installing it system wide.

So I have a layout like this:

. CMakeLists.txt
|_ src
|_ include
|_ third_party
   |_ my_library
      |_ CMakeLists.txt
      |_ src
      |_ include

Now the CMakeLists.txt in my_library creates some lib with add_library(), and it also has an install target which copies libmy_library.a to my_library/lib.

I would like to depend on this library in my toplevel CMakeLists.txt. So what I need is for the my_library CMake project to be built and installed before I try to reference the library for my own targets. I tried using ExternalProject_Add() for this like so:

./CMakeLists.txt

set(MY_LIB_DIR ${CMAKE_SOURCE_DIR}/third_party/my_library)

ExternalProject_Add(my_library
    SOURCE_DIR ${MY_LIB_DIR}
    UPDATE_DISCONNECTED TRUE 
    BUILD_IN_SOURCE FALSE
    BINARY_DIR ${MY_LIB_DIR}/build
)

list(APPEND CMAKE_PREFIX_PATH ${MY_LIB_DIR}/lib)

# find compiled lib
find_library(MY_LIB_LIBRARY my_library)

add_executable(main ${SOURCES})
add_dependencies(main my_library)  
target_link_libraries(main ${MY_LIB_LIBRARY})

However, this does not work since at the time cmake reads the configuration, libmy_library.a has not been built yet.

The question thus: How can I make CMake build and install an external project so I can list install artifacts as dependencies?

It has been suggested this is a duplicate, but the linked answer builds the external project inside the current project, which I find undesirable.