Building webpack for production - minified dependencies

Question

Say there is a module on npm called "awesomepackage". I can register it as a dependency of my application via package.json like so:

npm i --save awesomepackage

Inspecting my node_modules folder I see a folder named "awesomepackage" which looks like this:

- index.js
- package.json
- README.md
- lib/
    - awesomepackage.min.js

and I can use the package inside my own application like so:

import {AwesomeThing} from 'awesomepackage';

My question:

When I build my application using webpack my final bundle.js always contains the source code from node_modules/awesomepackage/index.js - regardless of any environment variables I have setup (eg NODE_ENV=production).

Is there any way to tell webpack to use the pre-built files (eg node_modules/awesomepackage/lib/awesomepackage.min.js) if they are shipped as part of an NPM package, instead of re-building from the source?

I have seen solutions that use resolve.alias and noParse in the webpack config with hardcoded paths to minified source files, but doesn't that mostly defeat the purpose of using npm and webpack? I don't want to have to hard-code and maintain paths to every dependency's minified build files!

I got here because my bundle.js jumped 2.5mb when I added moment.js - which is 167kb minified, including all locales, but of which the source code includes 2mb of tests.

Edit

Clearly there is more to it than just minifying the final bundle.js file at the end. For example: why would anybody want to include hundreds of unnecessary unit tests for dependencies in final production code? They don't. Which is why most libraries provide a dist (or similar) folder as part of their npm module.

Use `webpack.DefinePlugin` to set `process.env.NODE_ENV`, which will end to the development code (which uses the mentioned env variable) like `if (process.env.NODE_ENV !== 'production')` will be evaluated to `if (false)` statically and then removed by minifier as unreachable code. — zerkms, Dec 23 '15 at 02:48
As of moment - the inflation is caused by loading all locales. If you don't need all of them - load just necessary following the advice from http://stackoverflow.com/q/25384360/251311 — zerkms, Dec 23 '15 at 02:49
"why would anybody want to include hundreds of unnecessary unit tests for dependencies in final production code" --- no one does that indeed. — zerkms, Dec 23 '15 at 02:50
@zerkms (1) Setting process.env.NODE_ENV does not cause webpack to load minified dependencies. (2) If the inflation was due to the inclusion of locales, I would see [167kb](https://github.com/moment/moment/blob/develop/min/moment-with-locales.min.js) of inflation, but I see over 2mb - roughly the size of their `tests.js` file (3). — Alex McMillan, Dec 23 '15 at 03:26
1. It does, if the libraries implemented with that in mind (like reactjs). If not - it's minifier job to minify, webpack is just a loader. Normally it should not matter whether you load sources and minify them yourself or the dist. The former actually might be better since you use the never minifier version. 2. That's weird - I'm using momentjs and for sure there are no tests in my dev and prod builds. — zerkms, Dec 23 '15 at 03:52

Everettss · Accepted Answer · 2016-06-18T21:04:20.333

Let's consider your moment package.

You can always use

import moment from 'node_modules/moment/min/moment-with-locales.min.js'

But it's not a good idea.

You shouldn't use minified files in webpack

If you use

import moment from 'moment';

it will build module from source. But if you have minification process in webpack config file it shouldn't be much bigger than shipped .min.js files. Often your own build can be even smaller than original .min.js!

Experiment for moment

I measure additional size of my bundle.js after adding moment and its 190 kB and original moment/min/moment-with-locales.min.js size is 207 kB. Your 2.5 MB is probably that you add maps (in my configuration with maps moment weights 2.7 MB) You can analyze that it is build with all locales - webpack can print this statistics (sizes are before minification):

 [373] ./~/moment/moment.js 135 kB {0} [built]
 [374] ./~/moment/locale/af.js 2.62 kB {0} [optional] [built]
 [375] ./~/moment/locale/ar-ma.js 2.12 kB {0} [optional] [built]
 [376] ./~/moment/locale/ar-sa.js 3.16 kB {0} [optional] [built]
 [377] ./~/moment/locale/ar-tn.js 1.97 kB {0} [optional] [built]
 [378] ./~/moment/locale/ar.js 4.59 kB {0} [optional] [built]
 [379] ./~/moment/locale/az.js 3.35 kB {0} [optional] [built]
 [380] ./~/moment/locale/be.js 4.86 kB {0} [optional] [built]
 [381] ./~/moment/locale/bg.js 3.1 kB {0} [optional] [built]
 [382] ./~/moment/locale/bn.js 3.62 kB {0} [optional] [built]
 [383] ./~/moment/locale/bo.js 3.87 kB {0} [optional] [built]
 [384] ./~/moment/locale/br.js 3.44 kB {0} [optional] [built]
 [385] ./~/moment/locale/bs.js 4.7 kB {0} [optional] [built]
 [386] ./~/moment/locale/ca.js 2.97 kB {0} [optional] [built]
 [387] ./~/moment/locale/cs.js 6.4 kB {0} [optional] [built]
 [388] ./~/moment/locale/cv.js 2.33 kB {0} [optional] [built]
 [389] ./~/moment/locale/cy.js 2.92 kB {0} [optional] [built]
 [390] ./~/moment/locale/da.js 2.07 kB {0} [optional] [built]
 [391] ./~/moment/locale/de-at.js 3.03 kB {0} [optional] [built]
 [392] ./~/moment/locale/de.js 2.95 kB {0} [optional] [built]
 [393] ./~/moment/locale/dv.js 2.69 kB {0} [optional] [built]
 [394] ./~/moment/locale/el.js 3.73 kB {0} [optional] [built]
 [395] ./~/moment/locale/en-au.js 2.32 kB {0} [optional] [built]
 [396] ./~/moment/locale/en-ca.js 2.2 kB {0} [optional] [built]
 [397] ./~/moment/locale/en-gb.js 2.38 kB {0} [optional] [built]
 [398] ./~/moment/locale/en-ie.js 2.37 kB {0} [optional] [built]
 [399] ./~/moment/locale/en-nz.js 2.32 kB {0} [optional] [built]
 [400] ./~/moment/locale/eo.js 2.67 kB {0} [optional] [built]
 [401] ./~/moment/locale/es.js 2.97 kB {0} [optional] [built]
 [402] ./~/moment/locale/et.js 3.14 kB {0} [optional] [built]
 [403] ./~/moment/locale/eu.js 2.39 kB {0} [optional] [built]
 [404] ./~/moment/locale/fa.js 3.27 kB {0} [optional] [built]
 [405] ./~/moment/locale/fi.js 3.86 kB {0} [optional] [built]
 [406] ./~/moment/locale/fo.js 2.1 kB {0} [optional] [built]
 [407] ./~/moment/locale/fr-ca.js 2.09 kB {0} [optional] [built]
 [408] ./~/moment/locale/fr-ch.js 2.25 kB {0} [optional] [built]
 [409] ./~/moment/locale/fr.js 2.24 kB {0} [optional] [built]
 [410] ./~/moment/locale/fy.js 2.64 kB {0} [optional] [built]
 [411] ./~/moment/locale/gd.js 2.63 kB {0} [optional] [built]
 [412] ./~/moment/locale/gl.js 2.77 kB {0} [optional] [built]
 [413] ./~/moment/locale/he.js 3.41 kB {0} [optional] [built]
 [414] ./~/moment/locale/hi.js 3.88 kB {0} [optional] [built]
 [415] ./~/moment/locale/hr.js 4.83 kB {0} [optional] [built]
 [416] ./~/moment/locale/hu.js 4 kB {0} [optional] [built]
 [417] ./~/moment/locale/hy-am.js 3.22 kB {0} [optional] [built]
 [418] ./~/moment/locale/id.js 2.94 kB {0} [optional] [built]
 [419] ./~/moment/locale/is.js 4.43 kB {0} [optional] [built]
 [420] ./~/moment/locale/it.js 2.46 kB {0} [optional] [built]
 [421] ./~/moment/locale/ja.js 2.21 kB {0} [optional] [built]
 [422] ./~/moment/locale/jv.js 2.94 kB {0} [optional] [built]
 [423] ./~/moment/locale/ka.js 3.15 kB {0} [optional] [built]
 [424] ./~/moment/locale/kk.js 2.68 kB {0} [optional] [built]
 [425] ./~/moment/locale/km.js 2.02 kB {0} [optional] [built]
 [426] ./~/moment/locale/ko.js 2 kB {0} [optional] [built]
 [427] ./~/moment/locale/ky.js 2.69 kB {0} [optional] [built]
 [428] ./~/moment/locale/lb.js 4.92 kB {0} [optional] [built]
 [429] ./~/moment/locale/lo.js 2.32 kB {0} [optional] [built]
 [430] ./~/moment/locale/lt.js 4.37 kB {0} [optional] [built]
 [431] ./~/moment/locale/lv.js 3.9 kB {0} [optional] [built]
 [432] ./~/moment/locale/me.js 4.1 kB {0} [optional] [built]
 [433] ./~/moment/locale/mk.js 3.11 kB {0} [optional] [built]
 [434] ./~/moment/locale/ml.js 2.81 kB {0} [optional] [built]
 [435] ./~/moment/locale/mr.js 5.39 kB {0} [optional] [built]
 [436] ./~/moment/locale/ms-my.js 2.86 kB {0} [optional] [built]
 [437] ./~/moment/locale/ms.js 2.85 kB {0} [optional] [built]
 [438] ./~/moment/locale/my.js 2.73 kB {0} [optional] [built]
 [439] ./~/moment/locale/nb.js 2.23 kB {0} [optional] [built]
 [440] ./~/moment/locale/ne.js 3.81 kB {0} [optional] [built]
 [441] ./~/moment/locale/nl.js 2.63 kB {0} [optional] [built]
 [442] ./~/moment/locale/nn.js 2.09 kB {0} [optional] [built]
 [443] ./~/moment/locale/pa-in.js 4.01 kB {0} [optional] [built]
 [444] ./~/moment/locale/pl.js 3.94 kB {0} [optional] [built]
 [445] ./~/moment/locale/pt-br.js 2.23 kB {0} [optional] [built]
 [446] ./~/moment/locale/pt.js 2.34 kB {0} [optional] [built]
 [447] ./~/moment/locale/ro.js 2.61 kB {0} [optional] [built]
 [448] ./~/moment/locale/ru.js 7.08 kB {0} [optional] [built]
 [449] ./~/moment/locale/se.js 2.23 kB {0} [optional] [built]
 [450] ./~/moment/locale/si.js 2.43 kB {0} [optional] [built]
 [451] ./~/moment/locale/sk.js 5.49 kB {0} [optional] [built]
 [452] ./~/moment/locale/sl.js 6.16 kB {0} [optional] [built]
 [453] ./~/moment/locale/sq.js 2.45 kB {0} [optional] [built]
 [454] ./~/moment/locale/sr-cyrl.js 4.1 kB {0} [optional] [built]
 [455] ./~/moment/locale/sr.js 4.09 kB {0} [optional] [built]
 [456] ./~/moment/locale/ss.js 3.12 kB {0} [optional] [built]
 [457] ./~/moment/locale/sv.js 2.4 kB {0} [optional] [built]
 [458] ./~/moment/locale/sw.js 2.08 kB {0} [optional] [built]
 [459] ./~/moment/locale/ta.js 4.21 kB {0} [optional] [built]
 [460] ./~/moment/locale/te.js 3.09 kB {0} [optional] [built]
 [461] ./~/moment/locale/th.js 2.4 kB {0} [optional] [built]
 [462] ./~/moment/locale/tl-ph.js 2.14 kB {0} [optional] [built]
 [463] ./~/moment/locale/tlh.js 4.09 kB {0} [optional] [built]
 [464] ./~/moment/locale/tr.js 2.9 kB {0} [optional] [built]
 [465] ./~/moment/locale/tzl.js 3.59 kB {0} [optional] [built]
 [466] ./~/moment/locale/tzm-latn.js 2.11 kB {0} [optional] [built]
 [467] ./~/moment/locale/tzm.js 2.06 kB {0} [optional] [built]
 [468] ./~/moment/locale/uk.js 5.4 kB {0} [optional] [built]
 [469] ./~/moment/locale/uz.js 2.03 kB {0} [optional] [built]
 [470] ./~/moment/locale/vi.js 2.71 kB {0} [optional] [built]
 [471] ./~/moment/locale/x-pseudo.js 2.5 kB {0} [optional] [built]
 [472] ./~/moment/locale/zh-cn.js 4.35 kB {0} [optional] [built]
 [473] ./~/moment/locale/zh-tw.js 3.12 kB {0} [optional] [built]
 [474] ./~/moment/locale ^\.\/.*$ 2.41 kB {0} [optional] [built]

So why your custom build is smaller? You can use more aggressive minification. My build give me this information (probably it loose some weight at this point):

Condition always true [./~/moment/moment.js:8,0]
Condition always true [./~/moment/locale/af.js:6,0]
Condition always true [./~/moment/locale/ar-ma.js:7,0]
Condition always true [./~/moment/locale/ar-sa.js:6,0]
Condition always true [./~/moment/locale/ar-tn.js:5,0]
Condition always true [./~/moment/locale/ar.js:8,0]
Condition always true [./~/moment/locale/az.js:6,0]
Condition always true [./~/moment/locale/be.js:8,0]
Condition always true [./~/moment/locale/bg.js:6,0]
Condition always true [./~/moment/locale/bn.js:6,0]
Condition always true [./~/moment/locale/bo.js:6,0]
Condition always true [./~/moment/locale/br.js:6,0]
Condition always true [./~/moment/locale/bs.js:7,0]
Condition always true [./~/moment/locale/ca.js:6,0]
Condition always true [./~/moment/locale/cs.js:6,0]
Dropping unreachable code [./~/moment/locale/cs.js:31,0]
Dropping unreachable code [./~/moment/locale/cs.js:40,0]
Dropping unreachable code [./~/moment/locale/cs.js:49,0]
Dropping unreachable code [./~/moment/locale/cs.js:58,0]
Dropping unreachable code [./~/moment/locale/cs.js:67,0]
Condition always true [./~/moment/locale/cv.js:6,0]
Condition always true [./~/moment/locale/cy.js:6,0]
Condition always true [./~/moment/locale/da.js:6,0]
Condition always true [./~/moment/locale/de-at.js:9,0]
Condition always true [./~/moment/locale/de.js:8,0]
Condition always true [./~/moment/locale/dv.js:6,0]
Condition always true [./~/moment/locale/el.js:6,0]
Condition always true [./~/moment/locale/en-au.js:5,0]
Condition always true [./~/moment/locale/en-ca.js:6,0]
Condition always true [./~/moment/locale/en-gb.js:6,0]
Condition always true [./~/moment/locale/en-ie.js:6,0]
Condition always true [./~/moment/locale/en-nz.js:5,0]
Condition always true [./~/moment/locale/eo.js:8,0]
Condition always true [./~/moment/locale/es.js:6,0]
Condition always true [./~/moment/locale/et.js:7,0]
Condition always true [./~/moment/locale/eu.js:6,0]
Condition always true [./~/moment/locale/fa.js:6,0]
Condition always true [./~/moment/locale/fi.js:6,0]
Condition always true [./~/moment/locale/fo.js:6,0]
Condition always true [./~/moment/locale/fr-ca.js:6,0]
Condition always true [./~/moment/locale/fr-ch.js:6,0]
Condition always true [./~/moment/locale/fr.js:6,0]
Condition always true [./~/moment/locale/fy.js:6,0]
Condition always true [./~/moment/locale/gd.js:6,0]
Condition always true [./~/moment/locale/gl.js:6,0]
Condition always true [./~/moment/locale/he.js:8,0]
Condition always true [./~/moment/locale/hi.js:6,0]
Condition always true [./~/moment/locale/hr.js:6,0]
Condition always true [./~/moment/locale/hu.js:6,0]
Dropping unused variable suffix [./~/moment/locale/hu.js:16,0]
Condition always true [./~/moment/locale/hy-am.js:6,0]
Condition always true [./~/moment/locale/id.js:7,0]
Condition always true [./~/moment/locale/is.js:6,0]
Condition always true [./~/moment/locale/it.js:7,0]
Condition always true [./~/moment/locale/ja.js:6,0]
Condition always true [./~/moment/locale/jv.js:7,0]
Condition always true [./~/moment/locale/ka.js:6,0]
Condition always true [./~/moment/locale/kk.js:6,0]
Condition always true [./~/moment/locale/km.js:6,0]
Condition always true [./~/moment/locale/ko.js:10,0]
Condition always true [./~/moment/locale/ky.js:6,0]
Condition always true [./~/moment/locale/lb.js:6,0]
Condition always true [./~/moment/locale/lo.js:6,0]
Condition always true [./~/moment/locale/lt.js:6,0]
Condition always true [./~/moment/locale/lv.js:7,0]
Condition always true [./~/moment/locale/me.js:6,0]
Condition always true [./~/moment/locale/mk.js:6,0]
Condition always true [./~/moment/locale/ml.js:6,0]
Condition always true [./~/moment/locale/mr.js:7,0]
Condition always true [./~/moment/locale/ms-my.js:6,0]
Condition always true [./~/moment/locale/ms.js:6,0]
Condition always true [./~/moment/locale/my.js:6,0]
Condition always true [./~/moment/locale/nb.js:7,0]
Condition always true [./~/moment/locale/ne.js:6,0]
Condition always true [./~/moment/locale/nl.js:6,0]
Condition always true [./~/moment/locale/nn.js:6,0]
Condition always true [./~/moment/locale/pa-in.js:6,0]
Condition always true [./~/moment/locale/pl.js:6,0]
Condition always true [./~/moment/locale/pt-br.js:6,0]
Condition always true [./~/moment/locale/pt.js:6,0]
Condition always true [./~/moment/locale/ro.js:7,0]
Condition always true [./~/moment/locale/ru.js:8,0]
Condition always true [./~/moment/locale/se.js:6,0]
Condition always true [./~/moment/locale/si.js:6,0]
Condition always true [./~/moment/locale/sk.js:7,0]
Dropping unreachable code [./~/moment/locale/sk.js:32,0]
Dropping unreachable code [./~/moment/locale/sk.js:41,0]
Dropping unreachable code [./~/moment/locale/sk.js:50,0]
Dropping unreachable code [./~/moment/locale/sk.js:59,0]
Dropping unreachable code [./~/moment/locale/sk.js:68,0]
Condition always true [./~/moment/locale/sl.js:6,0]
Condition always true [./~/moment/locale/sq.js:8,0]
Condition always true [./~/moment/locale/sr-cyrl.js:6,0]
Condition always true [./~/moment/locale/sr.js:6,0]
Condition always true [./~/moment/locale/ss.js:6,0]
Condition always true [./~/moment/locale/sv.js:6,0]
Condition always true [./~/moment/locale/sw.js:6,0]
Condition always true [./~/moment/locale/ta.js:6,0]
Condition always true [./~/moment/locale/te.js:6,0]
Condition always true [./~/moment/locale/th.js:6,0]
Condition always true [./~/moment/locale/tl-ph.js:6,0]
Condition always true [./~/moment/locale/tlh.js:6,0]
Condition always true [./~/moment/locale/tr.js:7,0]
Condition always true [./~/moment/locale/tzl.js:6,0]
Condition always true [./~/moment/locale/tzm-latn.js:6,0]
Condition always true [./~/moment/locale/tzm.js:6,0]
Condition always true [./~/moment/locale/uk.js:7,0]
Condition always true [./~/moment/locale/uz.js:6,0]
Condition always true [./~/moment/locale/vi.js:6,0]
Condition always true [./~/moment/locale/x-pseudo.js:6,0]
Condition always true [./~/moment/locale/zh-cn.js:7,0]
Condition always true [./~/moment/locale/zh-tw.js:6,0]

Reusing common modules

Moment isn't the best module for highlight the most powerful feature of webpack - reusing common modules (since moment doesn't use dependencies).

Lest's consider that your application use 3 modules and each of them have some dependencies.

                   size

module1            20 kB 
  dependencies
    - moduleA      10 kB 
    - moduleB      15 kB
    - moduleC      5  kB
module1.min        50 kB

module2            10 kB 
  dependencies
    - moduleA      10 kB 
    - moduleD      5  kB
module2.min        25 kB

module3            30 kB 
  dependencies
    - moduleA      10 kB 
    - moduleC      5  kB
module3.min        45 kB

If you will only use .min version of this 3 modules you will add 50 kB + 25 kB + 45 kB = 120 kB to your application.

                   size

module1            20 kB 
  dependencies
    - moduleA      10 kB  
    - moduleB      15 kB 
    - moduleC      5  kB 
module1.min        50 kB  <-- this

module2            10 kB  
  dependencies
    - moduleA      10 kB 
    - moduleD      5  kB 
module2.min        25 kB  <-- this

module3            30 kB  
  dependencies
    - moduleA      10 kB 
    - moduleC      5  kB
module3.min        45 kB  <-- this

On the other hand if you will build with source files, common modules will be added only once. Not like previously moduleA was included 3 times and moduleC 2 times. This times to your app will be added:

                   size

module1            20 kB  <-- this
  dependencies
    - moduleA      10 kB  <-- this
    - moduleB      15 kB  <-- this
    - moduleC      5  kB  <-- this
module1.min        50 kB

module2            10 kB  <-- this
  dependencies
    - moduleA      10 kB 
    - moduleD      5  kB  <-- this
module2.min        25 kB

module3            30 kB  <-- this
  dependencies
    - moduleA      10 kB 
    - moduleC      5  kB
module3.min        45 kB

And finally with this approach you add 95 kB to your application.

You spent a lot of time writing this out but you didn't show how you minified your source. Where is an example of your build? What you explain is all good but unless you are showing how you arrived at these reduced files it's worthless. — Craig, Mar 28 '20 at 01:30

score 1 · Answer 2 · answered Dec 23 '15 at 02:31

1

Why not just minify the entire webpack bundle.js? (source)

webpack -p for building once for production (minification)

Probably easier than trying to find some way to use the minified version of individual packages.

answered Dec 23 '15 at 02:31

Josh Beam

19,292
3
45
68

1

But then I am re-processing the entire source for every dependency! If this is the accepted method, why do most NPM packages include pre-built files? – Alex McMillan Dec 23 '15 at 02:34
They include the minified ones because lots of people use different environment setups. You can still `require('node_modules/someModule/minified.min.js')` or whatever directly. Why does it matter if you re-process the entire source for every dependency during the build process? Is this affecting your machine badly in some way? Also, 2.5mb is pretty tiny in the grand scheme of an application. I wouldn't be too concerned with micro-optimization if I were you, if it's not necessary. – Josh Beam Dec 23 '15 at 02:35
1

For example: moment.js includes a 167kb `moment/dist/moment-with-locales.min.js` file, but webpack uses `moment/moment.js` which pulls in 2mb worth of unnecessary unit tests! Obviously, the owner of the package provided the source for development where unit tests would be helpful, and also a distributable, minified, test-exclusive, optimized file for use in production. – Alex McMillan Dec 23 '15 at 02:37
Then I would probably just `require` the minified version. You can also set an `alias` in your `webpack-config` that looks for instances of, say, `momentjs`, and replaces that with the appropriate filepath to the minified version (as you suggested in your original question). Other than that, I'm not familiar with any other methods to do what you described. Perhaps someone else more well-informed than I can chime in on this. – Josh Beam Dec 23 '15 at 02:39
1

I understand; that is what I am doing. But it just doesn't make sense - I haven't seen any open source packages doing this... is it really industry standard to re-compile the entire source of every dependency every time your package is built, or to explicitly refer to (read: hardcode) the minified build file for every dependency? If so, why does anybody bother including the built/minified files, or even having a `dist` folder at all? – Alex McMillan Dec 23 '15 at 02:42
Built/minified files are still useful for those who are not using an environment setup such as `webpack`. In dev environments too, sometimes it's useful to have a minified version for testing. Other than that, I'm not sure. Maybe you can come up with something to solve this if there's not already something out there :) Not sure that's it's a huge problem though; re-processing entire modules takes very, very little computing power. If it was a major bottleneck, I'd understand, but I don't think it is. – Josh Beam Dec 23 '15 at 02:44
1

Please read my question again. I am more concerned with the final bundle containing unnecessary development code than "computing power". – Alex McMillan Dec 23 '15 at 02:46