48

I cannot update the git submodule, with the error:

$ git submodule init
Submodule 'build/html' (git@github.com:quadroid/clonejs.git) registered for path 'build/html'
...
$ git submodule update
Cloning into 'build/html'...
Warning: Permanently added 'github.com,207.97.227.239' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

But when I do the same tasks locally, everything is OK.

How do I fix this so that the Travis CI build passes and I can still click on the submodule in the repo to direct to it?

AntoineLB
  • 482
  • 3
  • 19
Quadroid
  • 825
  • 2
  • 9
  • 15
  • fixed by changing submodule url to git://github.com/quadroid/clonejs.git But now I can't push this module locally... – Quadroid Mar 28 '13 at 05:06
  • use a [deploy key](https://help.github.com/categories/30/articles) or have a script change the URL only in travis, not on your local – Nevik Rehnel Mar 28 '13 at 11:05
  • This article helped me https://awolski.com/sudden-permission-denied-publickey-error-in-travis/ In short, you have to disable and enable Travis builds for the repository. – Dmytro Dec 07 '18 at 15:19

6 Answers6

76

This can (thankfully) be easily solved by modifying the .gitmodules file on-the-fly on Travis, so that the SSH URL is replaced with the public URL, before initializing submodules. To accomplish this, add the following to .travis.yml:

# Handle git submodules yourself
git:
    submodules: false
# Use sed to replace the SSH URL with the public URL, then initialize submodules
before_install:
    - sed -i 's/git@github.com:/https:\/\/github.com\//' .gitmodules
    - git submodule update --init --recursive

Thanks to Michael Iedema for his gist from which I derived this solution.

If your submodules are private repositories, it should work to include credentials in the https URLs, I recommend making a GitHub access token with restricted permissions for this purpose:

# Replace <user> and <token> with your GitHub username and access token respectively
- sed -i 's/git@github.com:/https:\/\/<user>:<token>@github.com\//' .gitmodules
aknuds1
  • 65,625
  • 67
  • 195
  • 317
  • 5
    This is by far the best solution to this problem so far! – Stefan Dragnev Jul 09 '14 at 07:51
  • 1
    This doesn't seem to work for private repositories. All of these private repositories are under the same github account, and turned on for travis. Anyone has a clue? Thanks. – inder Jul 17 '14 at 17:59
  • @inder I haven't tried Travis with private repositories yet, but you'll need some way to give access to your repositories. It's kind of a problem in itself. – aknuds1 Jul 17 '14 at 19:04
  • @inder Can you access your private repositories by combining HTTPS with user/password, i.e., `https://user:password@github.com/organization/repo.git`? If so, you can modify the sed command correspondingly. Let me know if this works for you. See [Travis Pro documentation](http://docs.travis-ci.com/user/travis-pro/) for reference. – aknuds1 Jul 17 '14 at 19:16
  • @aknuds Yes, Travis Pro documentation works. I created a build user on GitHub and added its ssh keys as suggested in the link. Also, DELETE the submodule hack suggested above. Thanks. – inder Jul 17 '14 at 20:11
  • 2
    If you're on a Mac OS X system and you're wondering why sed fails, it's due macs specific sed version which required the -e option after the -i option. – Evils Dec 15 '14 at 21:11
  • I prefer to use another delimiter than `/` when replacing in URLs etc. `sed -i 's|git@github.com:|https://github.com/|' .gitmodules` – Matthias Kuhn Jan 18 '16 at 14:38
  • I agree this is a great solution, you can see the screenshot of it in action here https://twitter.com/DinisCruz/status/729580391686934528 – Dinis Cruz May 09 '16 at 07:55
  • @carlin.scott I wonder if Travis has changed its behaviour then. I would check with them. – aknuds1 Jul 04 '16 at 21:39
24

I'd recommend using the https scheme for submodules, as that'll allow you to pull on Travis and push locally: https://github.com/quadroid/clonejs.git.

sarahhodne
  • 9,796
  • 3
  • 39
  • 44
13

Travis now supports accessing submodule using ssh, which is by far the easiest solution. You only need to associate your ssh key (or the ssh key of a dedicated CI user) with the Github project you are building, as described in the documentation for private dependencies.

$ travis sshkey --upload ~/.ssh/id_rsa -r myorg/main

Note that Travis recommends creating a dedicated user so that you do not have to use your own ssh key.

emidander
  • 2,383
  • 22
  • 29
  • 1
    To make this explanation even more complete; find a user with access to both the repository Travis is building and that of the submodule (Travis recommends creating a specific CI user for this), and then run this command *for the repo doing the build*. – thoutbeckers Sep 04 '14 at 16:46
  • 1
    It's not available on travis.org (only travis.com, not free) – Toilal Sep 24 '14 at 04:46
  • This doesn't really improve security does it... having your github key spread around.. – matanster Oct 29 '15 at 13:17
  • 1
    @matt I updated my answer to indicate that Travis recommends using a dedicated CI user for this - which I think is a good idea for several reasons. – emidander Nov 02 '15 at 13:58
7

You get this error because you specified your submodules via ssh-urls. For ssh access from the travis-ci environment you need to configure a key.

Alternatively, you can just use relative URLs for your git submodules since you project and your submodules are all available on Github.

Git resolves relative urls against the ORIGIN.

Example:

Using the first 2 entries from your .gitmodules:

[submodule "lib/es5-shim"]
        path = lib/es5-shim
        url = git@github.com:kriskowal/es5-shim.git
[submodule "build/html"]
        path = build/html
        url = git@github.com:quadroid/clonejs.git

Replaced with relative URLs:

[submodule "lib/es5-shim"]
        path = lib/es5-shim
        url = ../../kriskowal/es5-shim.git
[submodule "build/html"]
        path = build/html
        url = ../clonejs.git

Then when cloning - say - via https the origin is set like this:

$ git clone https://github.com/quadroid/clonejs.git
$ cd clonejs
$ git remote -v
origin  https://github.com/quadroid/clonejs.git (fetch)
origin  https://github.com/quadroid/clonejs.git (push)

When cloning via ssh:

$ git clone git@github.com:quadroid/clonejs.git
$ cd clonejs
$ git remote -v                                
origin  git@github.com:quadroid/clonejs.git (fetch)
origin  git@github.com:quadroid/clonejs.git (push)

With relative urls, the usual submodule sequence works independently of the origin:

$ git submodule init
$ git submodule update
maxschlepzig
  • 35,645
  • 14
  • 145
  • 182
  • The relative URL's work very well in my case, and that is possibly the most elegant solution for public repos. Thanks for coming up with this technique! – matanster Oct 24 '15 at 01:50
  • 1
    Annoyingly however, it seems to break github's automatic hyperlinking to the submodules github page. – matanster Oct 24 '15 at 02:02
  • This doesn't work for private submodules. It seems that travis hides the fact that it's using a private token to access the repo but it doesn't do this when cloning submodules. – carlin.scott Oct 17 '17 at 23:51
1

You can also just directly manipulate your .gitmodules file via git. (Inspired by this answer).

git config --file=.gitmodules submodule.SUBMODULE_PATH.url https://github.com/ORG/REPO.git
Community
  • 1
  • 1
loudmouth
  • 1,946
  • 1
  • 12
  • 16
1

Actual on 2022:

  1. Go to the repo settings in travis-ci and enable Clone or Import option for repos which used as submodule Clone or Import

  1. Use https protocol in your .gitmodules (but better to use relative paths without a protocol)

  2. Enjoy. I spent only 3 hours to figure it out

Hett
  • 3,484
  • 2
  • 34
  • 51