TL;DR - It will be a lot of work. This is not something that the standard Android APIs support.
Here are a couple of approaches that may work
Approach #1
You will need to do is build and maintain a central database of all makes and models of Android phones and the latest available OS version for each one.
When your app needs to check if the OS is up to date it does the following:
Lookup the device's make and model, and its OS version. This Q&A explains how to get the current OS version:
Query the central database (running on your infrastructure) to find out what the latest OS version for the device is.
If the user's device is out of date, send the user the notification.
Downside: you will have to assemble the database of devices and OS versions, and keep it up to date. And you will need to pay for the infrastructure that hosts the database and responds to queries.
Approach #2
For each Android device manufacturer, research how their device firmware works out if there is an update available. Apparently, this functionality is not standardized. There are no standard Android APIs that will tell you if the user's device has an OS update available.
Downside: you have a lot of research and (possibly) implementation work to do to support a range of devices. It will quite possibly entail getting hold of a lot of different types of devices for testing. (You are unlikely to be able to develop and test on an emulator.