npm install not picking up the correct ssh keys and unable to install private repo via ssh

Question

I'm setting up my dev environment on my new machine (m2 Mac) and have setup the ~/.gitconfig ~/.gitconfig-work and ~/.ssh/config to separate my ssh keys for personal and professional use.

~/.gitconfig

[user]
  name = personal-username
  email = personal-email@gmail.com
  IdentityFile = ~/.ssh/id_rsa

[includeIf "gitdir:~/Documents/work/"]
  path = ~/.gitconfig-work

[init]
    defaultBranch = main

~/.gitconfig-work

[user]
name = work-username
email = work-email@work.com
IdentityFile = ~/.ssh/id_rsa_work

~/.ssh/config

Host github.com-personal
   HostName github.com
   User git
   IdentityFile ~/.ssh/id_rsa
   IdentitiesOnly yes

# Work
Host github-work
   HostName github.com
   User git
   IdentityFile ~/.ssh/id_rsa_work
   IdentitiesOnly yes

I add the keys to the ssh agent and cloning works fine provided I update the remote url to have the correct host field e.g git clone git@github-work:work/work-repo.git

Now the company that I work for have some npm packages that are the base of majority of our projects, simply running npm install would always work on my previous mac, but now when running npm install I get the following error

npm install

npm ERR! code 128
npm ERR! An unknown git error occurred
npm ERR! command git --no-replace-objects ls-remote ssh://git@github.com/work/work-private-repo.git
npm ERR! ERROR: Repository not found.
npm ERR! fatal: Could not read from remote repository.
npm ERR!
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.

npm ERR! A complete log of this run can be found in:

Im using node v16.16.0 & npm v8.11.0, have tried with node 14 & 12 and neither worked.

Im trying to figure out the difference between the dev setup from both machines but haven't found any differences.

From the error the issue is when trying to call

git --no-replace-objects ls-remote ssh://git@github.com/work/work-private-repo.git

When I update the remote call to

git --no-replace-objects ls-remote ssh://git@github-work/work/work-private-repo.git

to match my work profile and keys, it will return a valid response.

The package.json entry for the repo which can't be found is

"work-private-repo": "git+https://github.com/work/work-private-repo.git",

Answer 1

TL;DR

I believe you want to configure insteadOf URL tricks. In particular, you'll want:

git config --file ~/.gitconfig-work url.https://github-work/.insteadOf https://github.com/

for instance. (Double check all this, it's easy to get these wrong and I might have something backwards. Also, feel free to edit the files directly rather than using git config.) Drop the user.IdentityFile: it's not harmful but it will confuse people.

Long

Putting:

[user]
    IdentityFile = ~/.ssh/id_rsa_work

into a .gitconfig file will do absolutely nothing: no errors, no change in behavior of Git, no change in behavior of ssh. That's because Git allows you to set arbitrary unused variable names to arbitrary values, without complaint, and then it does nothing with those variables. user.IdentityFile is not used in Git.

The place you need to specify an IdentityFile (and preferably also set IdentitiesOnly yes) is in the .ssh/config file, so this part is completely correct:

Host github.com-personal
   HostName github.com
   User git
   IdentityFile ~/.ssh/id_rsa
   IdentitiesOnly yes

# Work
Host github-work
   HostName github.com
   User git
   IdentityFile ~/.ssh/id_rsa_work
   IdentitiesOnly yes

What you need next is on the Git side: you need to make a URL that contains github.com refer to github-work, from the second section here, or github.com-personal, from the first. (But the first is close enough to the defaults that it probably already works anyway: if you ask ssh to connect to github.com, there's no Host line, so you connect to github.com and use the default identities—including id_rsa.)

This is where the insteadOf settings come in. According to the git config documentation:

url.<base>.insteadOf

Any URL that starts with this value will be rewritten to start, instead, with <base>. In cases where some site serves a large number of repositories, and serves them with multiple access methods, and some users need to use different access methods, this feature allows people to specify any of the equivalent URLs and have Git automatically rewrite the URL to the best alternative for the particular user, even for a never-before-seen repository on the site. When more than one insteadOf strings match a given URL, the longest match is used.

By putting these URL rewrites into the per-work and per-home settings, you can automatically replace a standard style, HTTPS-oriented URL into one of your personalized-style ssh URLs, and choose the personalized URL based on the location of the clone in your file system.

Since it's all prefix matching, you may want an entry for ssh://github.com/ as well: as it is you're only telling Git to switch to ssh when using https://github.com/. This way you'll personalize ssh URLs as well as HTTPS ones.

See also How do I get git to default to ssh and not https for new repositories.