It is difficult to diagnose the precise problem you are having without hands-on to your entire code base and backend (as others have suggested, the problem may not be angular at all).
Having said that, I HIGHLY recommend you start using the angular-cli. It was designed by the angular team to accomplish all of the things you need to do in a easy-to-use command line interface. So my answer is predicated on the use of the angular-cli.
Here are the general things you can do to optimize your ng2 project for production:
1) Ahead of Time (AoT) Compilation - Bundling/Minification/Tree-shaking
Look, forget about the headache of configuring a bunch of gulp tasks to accomplish all of these things. The angular-cli handles Bundling/Minification/Tree-shaking/AOT compilation in one easy step:
ng build -prod -aot
This will produce the following js files in your "dist" folder:
inline.d41d8cd98f00b204e980.bundle.js
vendor.d41d8cd98f00b204e980.bundle.js
main.d41d8cd98f00b204e980.bundle.js
styles.4cec2bc5d44c66b4929ab2bb9c4d8efa.bundle.css
You will also get the gzipped versions of these files for MORE optimization:
inline.d41d8cd98f00b204e980.bundle.js.gz
vendor.d41d8cd98f00b204e980.bundle.js.gz
main.d41d8cd98f00b204e980.bundle.js.gz
Angular's AOT compilation will automatically do "tree shaking" on your code and get rid of any unused references. For example, you may use lodash in your project, but you probably only use a few lodash functions; tree shaking will trim away all the unused parts of lodash that are not needed in your final build. And most importantly, AOT compilation will pre-compile all of your code and views which means LESS time the browser needs to get the ng2 app rolling. Click here for more info on angular's AOT compilation.
2) Lazy loading parts of your app
If you further carve up your app into different parts, you do not need to load EVERYING when your app first loads. You can specify different modules for your application that then can be bundled (by the angular-cli aot compiler) into different chunks. Read up here to learn how to organize your project into modules that you can compile into chucks that are only loaded AS NEEDED. Angular-cli will manage the creation of these chunks for you.
3) Angular Universal
Now if you really want to make your load time EXTREMELY fast then you will want to consider implementing Angular Universal, which is when you compile your initial view ON THE SERVER. I have not used Angular Universal as I have been able to achieve fast load times with steps 1 and 2. But it is an exciting option in the ng2 toolset. Keep in mind you don't compile or run the ng2 app on the server, you compile the initial view serverside so the user quickly receives a jolt of html and thus the user PERCEIVES the load time to be very fast (even though a full load will still lag behind a little bit). This step does not obviate the need for the other steps. As a bonus, Angular Universal also is supposed to help with SEO.
4) Secondary bundling
If I am not using lazy loading, I usually go ahead and further bundle the distribution files that are generated from AOT compilation. Thus I create one main.bundle.js file that concats inline.js, vendor.js and main.js files. I use gulp for this.