requirejs optimizer - need help optimizing our application

Question

We have a large scale javascript application that we are trying to concatenate and minify using grunt-contrib-requirejs. We use the Aura framework. We use bower to pull in dependencies from other repositories into our main application.

Here is our app structure

- app/  
   |  
   |_ main.js  
   |_ index.html  
   |_ css
   |_ bower_components/  
      |_ core/
      |_ widget1/  
          |_ main.js  
          |_ views/  
          |_ models/  
      |_ widget2/  
      |_ extension1/  
      |_ extension2/  
      |_ other third party libraries, etc
- Gruntfile.js
- bower.json

main.js:

requirejs.config({
    shim: {
        ...
    },
    paths: {
       'core': 'bower_components/core/app/main',
       'aura': 'bower_components/aura/dist/',
       'jquery': 'bower_components/jquery/jquery',
       'backbone': ...,
       ... lots of other paths
    }
});
define(['core', 'jquery', 'jqueryui'], function(core, $) {
    // Here we start Aura, which finds our extensions and widgets in
    // bower_components
});  

Our current requirejs task config:

requirejs: {
    compile: {
        options: {
            mainConfigFile: 'app/main.js',
            name: 'main',
            include: ['bower_components/widget1/main', 'bower_components/widget2/main',
                'bower_components/extension1/main', 'bower_components/extension2/main'],
            exclude: ['jquery'],
            insertRequire: ['main'],
            out: 'dist/app.js'
        }
    }
}

This concats our app/main and its dependencies, but when we try to run it, we get errors like:
GET http://www.my-machine:8080/bower_components/underscore/underscore.js 404 (Not Found) even though underscore is a dependency of many of the widgets we include.

We have extensively tried different options in the r.js examples, and read through many stackover flow issues trying to find an answer.
We want advice on how to build this into one minified file with this structure:

UPDATE #2: Correct file structure

- dist/  
    |_ index.html // copied using grunt copy
    |_ css        // copied using grunt copy
    |_ app.js     // Built with grunt requirejs

UPDATE
We have included underscore in our shim which fixed the above error, but we're still getting another error:

Failed to load resource: the server responded with a status of 404 (Not Found)
http://my-machine.com:8080/bower_components/core/app/main.js  

This is included in the minimized file so I don't understand why it can't find that file:

define("bower_components/core/app/main.js", ["aura/aura", "bootstrap"], function(){
...
});

UPDATE #3
The define above came from the file generated by the optimization tool! The original define for that module, core/app/main.js, looks like:

define(['aura', 'bootstrap'], function(Aura) {
    ...
});

Answer 1

You mention having this in your optimized file:

define("bower_components/core/app/main.js", ["aura/aura", "bootstrap"], function(){
...
});

But this does not make sense. If I reproduce a structure similar to what you show in your question and run an optimization on it then the file located at bower_components/core/app/main.js is optimized as:

define("core", ...

because it is known as core to the rest of your application due to the paths setting 'core': 'bower_components/core/app/main'.

I can reproduce the incorrect module name if I add 'bower_components/core/app/main.js' to the include setting in the build configuration. When I add this name to the include list, I also get the loading error you mentioned. Make sure that your include list does not contain this name, and make sure that there is nothing anywhere which lists 'bower_components/core/app/main.js' as a dependency. This name should not appear in any define or require call.

Answer 2

A couple of considerations first:

First of all, you should avoid defining module with a name. The name is to be generated by the optimization tool normally. Please read here: http://requirejs.org/docs/api.html#modulename

Second, you should understand what the baseUrl is and when it's used to located resources. The baseUrl seems to be "app/" in your case although you have not defined it explicitly. You should read this also: http://requirejs.org/docs/api.html#config-baseUrl So if you use the ".js" extension, then the baseUrl won't be taken into account.

Now what you should try:

To start with, remove the first argument "bower_components/core/app/main.js" from define function. Then you should refer to this module by auto-generated module id that will be a path relative to the baseUrl without ".js" extension.