tl,dr;
To properly test client-side caching for a page, copy/paste the URL in a new tab instead of hitting F5.
why?
What happens when a user hits Refresh is not entirely up to you, as a developer. It's also up to the browser manufacturer.
If you're looking for a way to forbid a browser to load fresh data when it already has a version which, according to your cache-control server-side settings, is still valid, there isn't one.
It's up to the browser manufacturer to guess what the user really wants and to provide it. On average, when the guess is correct, their market share goes up. When wrong, it goes down. For obvious reasons, you have no say in it.
You should also consider the vast majority of site owners do not know how to tweak cache-control and don't bother paying someone to do it for them, so they just go with defaults. So, in order to keep their users happy, browser manufacturers need to guess correctly when the user wants the content refreshed, even if the server says it's fresh enough.
In other words, you can't enforce client side caching (but you can server-side!). You can only suggest it. If the browser manufacturer has reason to believe the user wants a "Clear cache and hard reload" but is not tech-savvy enough to perform it, they'll perform it on simple Refresh.
Chrome, based on large amounts of usage data, tries to guess what's the "primary cache" for a page and what is "secondary cache". It tries to balance providing fresh content (by wiping out primary cache for that page) with doing it in a timely manner (by not wiping out secondary cache). I believe any resource loaded by more than one page on the same domain is marked as "secondary" although I'd guess the algorithm is smarter than just that.
Firefox, as far as I know, doesn't have multiple types of refresh, so if the user hits refresh all cache for said page is gone.