The simplest method of deploying Qt for OS X is to use the macdeployqt command line tool, and you have identified correctly that the normal method is to place the frameworks inside the app bundle, but multiple apps will each have copies of the frameworks.
It is reasonable to suggest moving the Qt frameworks to a separate, external location and linking to that instead. However, you will need to manage the framework carefully, especially when it comes to providing updates and be aware that if the framework is removed or altered, all your applications will fail to load. This, however is the same for any framework dependent application.
The thing to consider is where to place the framework. Normally, external frameworks reside in /Library/Frameworks, but if we all start to use that for Qt, problems may occur when your app is installed and another developer installs their app's frameworks with a different version of the libraries.
Apple defines various 'key directories' for applications and initially, the most likely location would appear to be the "Application Support Directory", but the documentation states that this is for: -
any type of file that supports the app but is not required for the app to run
This location is often used for support files, such as templates for the user to select.
If your application is to be deployed via the Apple Store, I wouldn't be surprised if it is rejected if you use this location. However, you're not using the Apple Store, then you could deploy the frameworks here.
If the Apple Store is your method of deployment, then /Library/Frameworks is probably the only place acceptable for the Qt framework to reside, with the possibility of the problems I've mentioned above.
Alternatively, consider just how many applications you're developing and is it really an issue to bundle the frameworks multiple times against the advantages that it brings, such as allowing the user to cleanly remove the application and all of its dependencies, as well as reducing problems of the framework being altered or removed accidentally?
If you choose to move them externally, you can refer to the answer to this question, which comprehensively explains how to make installer packages, after having updated your binary dependencies on the frameworks with the install_name_tool, as outlined here.