From the docs. From what I read you can trigger optimisations and features with this setting, but it greatly depends on minSdkVersion
and maxSdkVersion
targetSdkVersion
An integer designating the API Level that the application targets. If not set, the default value equals that given to minSdkVersion.
This attribute informs the system that you have tested against the target version and the system should not enable any compatibility
behaviors to maintain your app's forward-compatibility with the target
version. The application is still able to run on older versions (down
to minSdkVersion).
As Android evolves with each new version, some behaviors and even appearances might change. However, if the API level of the platform is
higher than the version declared by your app's targetSdkVersion, the
system may enable compatibility behaviors to ensure that your app
continues to work the way you expect. You can disable such
compatibility behaviors by specifying targetSdkVersion to match the
API level of the platform on which it's running. For example, setting
this value to "11" or higher allows the system to apply a new default
theme (Holo) to your app when running on Android 3.0 or higher and
also disables screen compatibility mode when running on larger screens
(because support for API level 11 implicitly supports larger screens).
There are many compatibility behaviors that the system may enable based on the value you set for this attribute. Several of these
behaviors are described by the corresponding platform versions in the
Build.VERSION_CODES reference.
To maintain your application along with each Android release, you should increase the value of this attribute to match the latest API
level, then thoroughly test your application on the corresponding
platform version.
reference
Edit: (now that I got your question correctly I guess)
compileSdkVersion
The compileSdkVersion is the version of the API the app is compiled
against. This means you can use Android API features included in that
version of the API (as well as all previous versions, obviously). If
you try and use API 16 features but set compileSdkVersion to 15, you
will get a compilation error. If you set compileSdkVersion to 16 you
can still run the app on a API 15 device as long as your app's
execution paths do not attempt to invoke any APIs specific to API 16.
reference
So I would assume the resulting binary has some different pointers here and there.