node long path module names fail teamcity build

Question

We are trying to Install node in our asp.net MVC project how ever when we checked our code in it would fail the builds in team city. this is due the well known issue of the long module path names that NPM uses.

here is the log:

    [08:07:46]Checking for changes
[08:07:49]Publishing internal artifacts (5s)
[08:07:54][Publishing internal artifacts] Sending build.start.properties.gz file
[08:07:49]Clearing temporary directory: C:\TeamCity\buildagent3\temp\buildTmp
[08:07:54]Clean build enabled: removing old files from C:\TeamCity\buildagent3\work\57c6a27fa330ee2f
[08:07:54]Checkout directory: C:\TeamCity\buildagent3\work\57c6a27fa330ee2f
[08:07:54]Updating sources: agent side checkout (15s)
[08:07:54][Updating sources] Will perform clean checkout. Reason: Checkout directory is empty or doesn't exist
[08:07:54][Updating sources] Cleaning C:\TeamCity\buildagent3\work\57c6a27fa330ee2f
[08:07:54][Updating sources] VCS Root: git - tempsearch (15s)
[08:07:54][VCS Root: git - tempsearch] revision: cf23c64dd32077edeb1b96a85d1be104bd127044
[08:07:54][VCS Root: git - tempsearch] Cleaning C:\TeamCity\buildagent3\work\57c6a27fa330ee2f
[08:07:54][VCS Root: git - tempsearch] The .git directory is missing in 'C:\TeamCity\buildagent3\work\57c6a27fa330ee2f'. Running 'git init'...
[08:08:05][VCS Root: git - tempsearch] Checking out branch refs/heads/develop in git - tempsearch in C:\TeamCity\buildagent3\work\57c6a27fa330ee2f with revision cf23c64dd32077edeb1b96a85d1be104bd127044
[08:08:10]
[Updating sources] Failed to perform checkout on agent: '"C:\Program Files (x86)\Git\bin\git.exe" checkout -q -f develop' command failed.
stderr: fatal: cannot create directory at 'node_modules/grunt-contrib-jasmine/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/node_modules/fs-extra/node_modules/rimraf/node_modules/glob/node_modules/minimatch/node_modules/brace-expansion': No such file or directory
[08:08:10]Publishing internal artifacts
[08:08:10][Publishing internal artifacts] Sending build.finish.properties.gz file
[08:08:10]Build failed to start. Artifacts will not be published for this build
[08:08:10]Build finished

the error:

    Error while applying patch: Failed to perform checkout on agent: '"C:\Program Files (x86)\Git\bin\git.exe" checkout -q -f develop' command failed.
stderr: fatal: cannot create directory at 'node_modules/grunt-contrib-jasmine/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/node_modules/fs-extra/node_modules/rimraf/node_modules/glob/node_modules/minimatch/node_modules/brace-expansion': No such file or director

are there any long term solutions to this problem?

Answer 1

To the respond of the comment. I have a bit more space here ;).

It depends a bit on your exact situation how I would solve this. This is how I have set things up at the company here:

Setup for developing

Some node modules have to be installed globally, so each developer would have to install those globally (grunt-cli, karma, bower for example). The other node_modules that can/have to be installed locally you would add to a package.json file like:

      "devDependencies": {
            "grunt": "~0.4.2",
            "grunt-karma": "~0.7.3",
            "karma-jasmine": "~0.2.0",
            "karma-firefox-launcher": "~0.1",
            "karma-chrome-launcher": "~0.1",
            "karma-phantomjs-launcher": "~0.1",
            "grunt-contrib-clean": "~0.4.1",
            "grunt-contrib-copy": "~0.4.1",
            "grunt-contrib-jshint": "~0.10.0",
            "grunt-contrib-concat": "~0.3.0",
            "grunt-contrib-watch": "~0.5.3",
            "grunt-contrib-uglify": "~0.4.0",
            "grunt-contrib-less": "~0.9.0"
       }

More information about package.json you can find here http://browsenpm.org/package.json. Note that bower has to be installed globally and locally (if you are using it).

This means that every develop has to do a global install of the node_modules you use once. You might want to choose a specific version of each node_module so that everyone is using the same version. You can install global modules like this:

~ npm install grunt-cli karma bower -g

You add the package.json file with the local modules to the base structure of your project, and you add this to the git repository. Each developer that will do a checkout will receive this file. They will have to do an install of the local modules so that everyone is using the same tools:

~ npm install

This means that the local modules don't have to be part of your git repo, which I find good practice since it isn't source code. But it also solves your problem of git not supporting the long node_modules paths.

If you have a build process with Node setup that doesn't change anymore, the npm_modules installed both globally and locally won't have to change. If you change the tools, each develop will have to redo the npm install. But for me this hasn't happened yet.

Build server

You will also have to install node and these global modules on the build server so that you have access to them during the build. What I have done is create a .bat file which gets executed during the build which includes a 'npm install' so that it will resolve the node_modules during the build as well. This way you also have access to the latest tools during a build.

Multiple projects If you have multiple projects which use the same local node_modules for the build, you should take a look at grunt-collections. You can create one central location for the node_modules this way, so the developers only have to install them once for all the projects There is a really good post about this here:

Centralise node_modules in project with subproject

Hope this clears things ups

Answer 2

I had this error too but in my case the cause was using an outdated version of npm, v1.4.28.

Updating to npm v3 followed by rm -rf node_modules ; npm -i worked for me. npm issue 2697 has details of the 'maximally flat" folder structure included in npm v3 (released 2015-06-25).