183

I'm using a module for my angular app called angular-translate. However, I've had to make a few small modifications to the source code to get everything working the way I'd like, and now I want to persist those changes on npm install. A colleague suggested that I fork the repo of the source code and point to my forked repo as a dependency, which I've tried in these ways, e.g.

npm install https://github.com/myRepo/angular-translate
npm install https://github.com/myRepo/angular-translate/archive/master.tar.gz

The first gives me a directory like this with no build. Just a package.json, .npmignore, and some markdown files

-angular-translate
   .npmignore 
   .nvmrc
    CHANGELOG.md 
    package.json
    etc

The second npm install gives me the full repo, but again I don't get a build like when I use the command npm install angular-translate. I've seen some dicussion of running the prepublish script, but I'm not sure how to do this when installing all the modules. I've also tried publishing the fork as my own module to the npm registry, but again I get no build, and I'm not sure that's the right thing to do...

I apologise for my ignorance on the topic. I don't have a huge amount of experience with npm. Would love to get some feedback on this issue. It seems like it could be a common enough issue when modifications need to be made to a package's source code? Maybe there's a better solution?

kaya3
  • 47,440
  • 4
  • 68
  • 97
hughesjmh
  • 2,133
  • 2
  • 11
  • 11
  • 1
    This is caused by a wrongly configured repo on Github. It is missing a `.npmignore` or `package.json#files`. See [my answer](https://stackoverflow.com/a/57829251/4612476) for more details. – Cameron Tacklind May 29 '21 at 01:39

6 Answers6

197

Try npm install <ghusername>/<repoName>, where <ghUsername> is your GitHub username (without the @) and <repoName> is the name of the repository. That should correctly install it. You will most likely want to use the --save or --save-dev flag with the install command to save dependency in your package.json.

If that isn't working correctly, check the contents of your .npmignore file.

Don't panic if the install command takes a long time; installing from a git repository is slower than installing from the npm registry.

Edit:

Your problem is that in your case, dist/ is not committed to the repo (since it is in the .gitignore). That is where the actual code lives. dist/ is built from the files in src/ before the package is published to the npm registry, but dist/ is never committed to the repo.

It's ugly, but in this case you will have to remove dist/ from the .gitignore and then run:

npm run build
git add .
git commit
git push

(Ensure that you have run npm install first)

You then should be able to install from github.

There might be another way to do this using a prepare script, but I'm not sure if that's possible; I've never tried it. Edit: Cameron Tacklind has written an excellent answer detailing how to do this: https://stackoverflow.com/a/57829251/7127751

RyanZim
  • 6,609
  • 1
  • 27
  • 43
  • Thanks for the reply, Ryan. Gave that a try, but I got the same result as the 1st npm install. I then tried removing the .npmignore file and I get the whole repo, same as the second result. The problem still remains that the build isn't happening. – hughesjmh Nov 10 '16 at 14:23
  • 2
    Is there really no other way to do with other than unignore dist? I want to make a PR to the original repo, but with the dist, is he not gonna like it? – Ka Mok Dec 15 '17 at 18:30
  • 11
    @KaMok I think another option is to, in the `package.json` file of your fork, under `scripts`, rename `prepublish` to `prepare`. It seems when `npm install` or `npm install github:user_name/fork_name --save` (and same for yarn) is executed it also runs what's in the `prepare` script. This is assuming the `prepublish` script of this package creates the build files, which is usually the case. – davidfrancisco Feb 14 '18 at 17:17
  • 11
    and if you need a specific branch, `npm install /#branchName` – DrMeers Sep 03 '18 at 11:58
  • This is a very hacky solution. With rare exceptions, built files should never be included in repositories. See my answer for a better way with `prepare`. – Cameron Tacklind Sep 06 '19 at 22:47
  • @CameronTacklind In general, with rare exceptions, you should never install from git. The problem with `prepare` is that it requires listing build-time deps in `dependencies`, which is also a bad practice. It's just bad any way you look at it. – RyanZim Sep 07 '19 at 15:45
  • 2
    @RyanZim You are incorrect. Installing from git works very well as long as it is setup properly. https://docs.npmjs.com/cli/install "If the package being installed contains a `prepare` script, its `dependencies` and `devDependencies` will be installed, and the prepare script will be run, before the package is packaged and installed." – Cameron Tacklind Sep 07 '19 at 20:03
  • Someone made a [post](https://blaipratdesaba.com/how-to-use-an-npm-node-module-that-has-been-forked-b7dd522fdd08) about how we can install using post install script. Please look into this for reference, worked well for me. – Aman Relan Mar 09 '21 at 13:14
  • There is no `/dist` in my `.gitignore` – Zeeshan Ahmad Khalil Aug 03 '21 at 05:37
56

TL;DR use a prepare script

and don't forget package.json#files or .npmignore

Code published to npmjs.com is often not what's in the repository for the package. It is common to "compile" JavaScript source files into versions meant for general consumption in libraries. That's what's usually published to npmjs.com.

It is so common that it's a feature of npm to automatically run a "build" step before publishing (npm publish). This was originally called prepublish. It seems that Npm thought it would be handy to also run the prepublish script on an npm install since that was the standard way to initialize a development environment.

This ended up leading to some major confusion in the community. There are very long issues on Github about this.

In the end, in an effort to not change old behavior, they decided to add two more automatic scripts: prepublishOnly and prepare.

prepublishOnly does what you expect. It does not run on npm install. Many package maintainers just blindly switched to this.

But there was also this problem that people wanted to not depend on npmjs.com to distribute versions of packages. Git repositories were the natural choice. However it's common practice to not commit "compiled" files to git. That's what prepare was added to handle...

prepare is the correct way

If you have a repository with source files but a "build" step is necessary to use it,
prepare does exactly what you want in all cases (as of npm 4).

prepare: Run both BEFORE the package is packed and published, on local npm install without any arguments, and when installing git dependencies.

You can even put your build dependencies into devDependencies and they will be installed before prepare is executed.

Here is an example of a package of mine that uses this method.

Problems with .gitignore

There is one issue with this option that gets many people. When preparing a dependency, Npm and Yarn will keep only the files that are listed in the files section of package.json.

One might see that files defaults to all files being included and think they're done. What is easily missed is that .npmignore mostly overrides the files directive and, if .npmignore does not exist, .gitignore is used instead.

So, if you have your built files listed in .gitignore, like a sane person, and don't do anything else, prepare will seem broken.

If you fix files to only include the built files or add an empty .npmignore, you're all set.

My recommendation

Set files (or, by inversion, .npmignore) such that the only files actually published are those needed by users of the published package. Imho, there is no need to include uncompiled sources in published packages.

Original answer: https://stackoverflow.com/a/57503862/4612476

Cameron Tacklind
  • 5,764
  • 1
  • 36
  • 45
25

Update for those using npm 5:

As of npm@5, prepublish scripts are deprecated.

Use prepare for build steps and prepublishOnly for upload-only.

I found adding a "prepare": "npm run build" to scripts fixed all my problems.

Simon
  • 1,849
  • 3
  • 22
  • 29
  • That worked for me, too (in a fork I just made) — thanks! I wonder... why don't all packages just include this script config by default? Is it because package authors only consider the use case of installing their package from npm and not the case of installing it from a git repo? So they're just used to manually running `npm run build` and `npm run publish` and don't run into any problems or pain unless they someday try to install the repo via git? – Tyler Rick Aug 27 '19 at 23:08
  • There is one issue that gets many people when using `prepare`. See my answer for more details – Cameron Tacklind Oct 09 '20 at 01:37
8

Just use the command npm install git+https://git@github.com/myRepo/angular-translate.git. Thanks.

Bimal Jha
  • 391
  • 2
  • 14
5

To piggyback off of @RyanZim's excellent answer, postinstall is definitely a valid option for this.

Either do one of the following:

  1. Update the package.json in your forked repo to add a postinstall element to scripts. In here, run whatever you need to get the compiled output (Preferred).
  2. Update your package.json, and add a postinstall that updates the necessary directory in node_modules.

If you've forked another persons repository, then it might be worth raising an issue to illustrate the issue that installing their package through GitHub does not work as it does not provide the necessary means to build the script. From there, they can either accept a PR to resolve this with a postinstall, or they can reject it and you can do #2.

Mike B
  • 12,768
  • 20
  • 83
  • 109
  • Can you explain or have any resource on how to do #2? Which commands do I need to run after npm install when the gitrepo only contains src not dist – Daniel Jul 12 '18 at 08:12
  • 1
    @Daniel Apologies, I'm trying to remember the context of where I did this in the past. I think that by #2 I meant to add a postinstall step to your main packages.json that would run the steps for the top answer, so something along the lines of `"postinstall": "cd node_modules/scrape-twitter/ && npm install && npm run build"` to ensure that the package causing you trouble is built before your application runs. As said, though, I prefer the approach of forking the repo to my own user, adding the postinstall to the package itself, and using that in my application. – Mike B Jul 12 '18 at 13:45
  • I think adding to prepare should be preferred (@Simon's answer), because if installing from npm, you wouldn't need to do any additional building because it should already include a dist/ dir. See https://docs.npmjs.com/misc/scripts: "prepare: Run both BEFORE the package is packed and published, ... and when installing git dependencies" – Tyler Rick Aug 27 '19 at 23:05
  • `postinstall` is one step short of the right solution. Just use `prepare`. It's 2020. – Cameron Tacklind Jan 08 '20 at 00:11
-1

If you are using yarn like me. Imagine that you want to use a package like this: yarn add ghasemikasra39/gridfs-easy --save where the ghasemikasra39 is the username and gridfs-easy is the name of the repo

Kasra
  • 1,959
  • 1
  • 19
  • 29