Deploying complex node.js project with elastic beanstalk

Question

I wonder what is the best practice of deploying complex node.js with elastic beanstalk without relying on availability of external npm repository (and dealing with credentials and high availability of privately managed git repositories for internally developed packages).

It looks like there one school of thought which preaches to actually check in node_modules into the source tree for projects that are actually being deployed.

source 1: http://www.futurealoof.com/posts/nodemodules-in-git.html

source 2: http://eng.yammer.com/managing-node-js-dependencies-and-deployments-at-yammer/

So sounds like checking them in is the right approach, but then there is a problem of different binary formats for some compiled packages (developing on mac and deploying to linux)

I've tried doing as yammer guys suggested (checkin modules except the bin folders), but even then local "npm rebuild" command fails (it tries to chmod something in a bin folder that doesn't exist in express.js module) so I haven't even gotten to try to see what beanstalk default deployment environment will do with such a repository. I assume it runs "npm install" (which will do nothing), but will it run "npm rebuild"?

So, again, what is the best practice to deploy a a complex project with multiple dependencies? It must be a solved problem by now in the node/beanstalk world, isn't it?

Thanks

You might be using a lib that relies on a binary.If your staging env is 100% the same as your production env then go ahead,check in the node_modules folder,no question.You dont need npm rebuild if you are not using different versions of node in staging and in production.Just copy everything and you should be fine. — mpm, May 14 '14 at 05:41
mongodb bson, kerberos use binary and I am developing on a mac while deploying to an amazon linux machine, I can not commit binaries into the repository — Dmitry Fink, May 14 '14 at 07:11
I understand.but at this point you might want to think about using something like vagrant , it will make your life much more simple. — mpm, May 14 '14 at 07:17

score 2 · Answer 1 · edited May 23 '17 at 10:28

Here's my configuration that does what you're talking about. Save it in the .ebextensions folder and you'll be set. The only difference between mine and the superior answer in https://stackoverflow.com/a/23242623/34340 is the NPM_CONFIG_UNSAFE_PERM=true line, which I learned from https://forums.aws.amazon.com/thread.jspa?messageID=534612

packages:
  yum:
    git: []
    gcc: []
    make: []
    openssl-devel: []
    libxml2: []
    libxml2-devel: []

files:
  "/opt/elasticbeanstalk/env.vars" :
    mode: "000775"
    owner: root
    group: users
    content: |
      export HOME=/home/ec2-user # ADDED EXPORT COMMAND
      export NPM_CONFIG_LOGLEVEL=error
      export NPM_CONFIG_UNSAFE_PERM=true
      export NODE_PATH=`ls -td /opt/elasticbeanstalk/node-install/node-* | head -1`/bin
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh" :
    mode: "000775"
    owner: root
    group: users
    content: |
      #!/bin/bash
      . /opt/elasticbeanstalk/env.vars
      function error_exit
      {
        eventHelper.py --msg "$1" --severity ERROR
        exit $2
      }

      #install not-installed yet app node_modules
      if [ ! -d "/var/node_modules" ]; then
        mkdir /var/node_modules ;
      fi
      if [ -d /tmp/deployment/application ]; then
        ln -s /var/node_modules /tmp/deployment/application/
      fi

      OUT=$([ -d "/tmp/deployment/application" ] && cd /tmp/deployment/application && $NODE_PATH/npm install 2>&1) || error_exit "Failed to run npm install.  $OUT" $?
      echo $OUT
  "/opt/elasticbeanstalk/hooks/configdeploy/pre/50npm.sh" :
    mode: "000666"
    owner: root
    group: users
    content: |
       #no need to run npm install during configdeploy

I assume this solution makes the npm install faster, but it doesn't remove dependency on npmjs or other servers when deploying new instances, nor does it solve the problem of inconsistent module versions across instances, right? — Dmitry Fink, Aug 20 '14 at 03:42
That's correct - external services & inconsistent module versions can still impact you. For those situations, you'll likely want to move away from "git aws.push" towards something like https://github.com/simoneb/grunt-awsebtdeploy + a healthy helping of tight package.json versioning & npm shrinkwrap. — Eric Caron, Aug 21 '14 at 16:32

Deploying complex node.js project with elastic beanstalk

1 Answers1