5

I would like to deploy a nodejs project with frequent updates. npm is not available at the site so I must package the node_modules. This works ok but takes a long time to send to the customer over the available ftp connection (80MB of mostly node_module files every time). My workflow looks like this:

git clone project
npm install   # installs all my dev tools which I need for packaging
grunt build 
tar xvzf build.tar.gz build/

The build step minfifies my code packaging only what is needed. The node_modules folder is copied into the build folder. If I use npm install --production, I get a smaller footprint but miss the tools I need to build it in the first place. So in the end I go to some effort to make my code footprint small but all my work is undone by having to package such a large node_modules tree.

Is my approach wrong? Is there a simpler way to deploy where npm is not available on the production server or is there a good way to reduce the size of the node_modules folder?

chriskelly
  • 7,526
  • 3
  • 32
  • 50
  • 1
    Why not use differential deploy process? It's when you deploy to client only changed things. Files in the node_modules directory usually not so often changes. But it could be problem when you have only FTP access, in this case you will need to prepare differential release by yourself. Try to look at [grunt-newer](https://www.npmjs.org/package/grunt-newer). – raidendev Aug 29 '14 at 09:34
  • Thanks. I am using a differential approach but would like to simpify it. Thanks for pointing out grunt-newer. I will look at this and see if I can use it to package new modules. – chriskelly Aug 29 '14 at 10:25
  • 1
    On the top of what say @raidendev, you could also save bandwidth and faster your deploy by choosing a better compression algorithm. Like [LZMA](http://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Markov_chain_algorithm). – aymericbeaumet Aug 29 '14 at 16:51

1 Answers1

6

Update: Since writing this answer, npm3 (and yarn) arrived, and flattened npm dependencies. This reduces the size of the node_modules folder considerably (perhaps 20% - 30% for a typical project). Nevertheless, some of the tips below that will reduce your footprint by an order of magnitude.

I have compiled the list of findings for anyone wanting to

  1. deploy without npm on the server or
  2. reduce the footprint of the node_modules folder

Smaller node_modules footprint:

  • Use npm prune --production to remove devDependencies and purge additional modules

    In my case this node_modules folder size by about 20%.

    The bulk of large files under node_modules folders is confined to a small number of modules that are unused at runtime!. Purging/deleting these reduces the footprint by a factor of 10! e.g: karma, bower, less and grunt. Many of these are used by the modules themselves and have no place in a production build. The drawback is that npm install has to be run before each build.

  • Use partial npm packages

    Many npm packages are available in parts. For example, of installing all of async or lodash install only the bits you need: e.g.

Bad: npm install -save lodash async

Good: npm install --save async.waterfall async.parallel lodash.foreach

Typically, individual lodash modules are 1/100th the size of the full package.

  • npm-package-minifier may be used to reduce the size of the node_modules tree

    Compacting node_modules for client-side deployment

    This basically deletes a lot of unused files in the node_modules tree. This tool will reduce the size of devDependencies also so it should be run on a 'production' version of node_modules.

Reducing size of updates

  • Differential deployment

    As mentioned in the comments, updates may be split into updates where dependency changes are required or only business logic changes. I have tried this approach and it greatly reduces the footprint for most updates. However, it also increases the complexity of deployment.

Community
  • 1
  • 1
chriskelly
  • 7,526
  • 3
  • 32
  • 50