33

We are compiling Doxygen docs on the travis-ci server and want to push them onto our gh-pages branch.

How do I handle the authorization for git push? Does someone have an example for using encrypted variables in travis-ci? Should I go for https authorization or for an SSH key?

Stevoisiak
  • 23,794
  • 27
  • 122
  • 225
Stasik
  • 2,568
  • 1
  • 25
  • 44
  • Superset: http://stackoverflow.com/questions/12343452/how-to-publish-artifacts-in-travis-ci – Ciro Santilli OurBigBook.com Oct 13 '15 at 18:02
  • 2
    I found this site incredibly useful [Automatically Publish Javadoc to gh-pages with travis-ci](http://benlimmer.com/2013/12/26/automatically-publish-javadoc-to-gh-pages-with-travis-ci/) This is the script that I'm using to do this for my build: https://github.com/WPIRoboticsProjects/GRIP/blob/master/.travis-scripts/push-javadoc-to-gh-pages.sh – Jonathan Leitschuh Nov 30 '15 at 19:29
  • What if we just want to publish compiled binaries to an existing GitHub Pages? – Stevoisiak Mar 29 '17 at 02:27
  • @StevenVascellarosame procedure, as with .html files – Stasik Mar 29 '17 at 10:24

5 Answers5

43

Step-by-step example with HTTPS API Token in environment variable

Others have mentioned it, but here goes a more detailed procedure.

  1. Create a separate repository for the website (optional). This will reduce the likelihood that you overwrite your main repository, and will keep output files from polluting it.

  2. Get a Personal Access Token under https://github.com/settings/tokens

    Only enable "public_repo" access for public repositories, "repo" for private.

    Save the token somewhere as you can only see it once.

  3. On the Travis settings for the repository https://travis-ci.org/<me>/<myrepo>/settings create an environment variable:

    GITHUB_API_KEY=<token>

    and make sure to mark "Display value in build log" as "Off".

    This is safe because only authorized pushes by you see such environment variables, so if a malicious user tries to make a pull request to get your string, the variable won't be there.

    Just make sure that you never, ever list your environment variables on your build!

  4. Add the following to your .travis.yml:

    after_success: |
  if [ -n "$GITHUB_API_KEY" ]; then
    cd "$TRAVIS_BUILD_DIR"
    # This generates a `web` directory containing the website.
    make web
    cd web
    git init
    git checkout -b gh-pages
    git add .
    git -c user.name='travis' -c user.email='travis' commit -m init
    # Make sure to make the output quiet, or else the API token will leak!
    # This works because the API key can replace your password.
    git push -f -q https://<me>:$GITHUB_API_KEY@github.com/<me>/<myrepo>-gh-pages gh-pages &>/dev/null
    cd "$TRAVIS_BUILD_DIR"
  fi

Alternative travis encrypt method

Explained in detail at: https://stackoverflow.com/a/33109519/895245

Encrypt the string GITHUB_API_KEY=<key> with the travis gem, and add it to your .travis.yml:

env:
  secure: <encrypted>

This has the advantage that it does not require using the Travis web interface, but does require using a Gem and some more copy pasting.

Ciro Santilli OurBigBook.com
  • 347,512
  • 102
  • 1,199
  • 985
  • This isn't helpful in the regard of pushing to a repository on another server (i.e. not GitHub). – IIllIIll Dec 24 '15 at 01:30
  • 2
    @Arcrammer sure. But how could this question be interpreted for other servers? E.g.: does any other service support Travis CI? – Ciro Santilli OurBigBook.com Dec 24 '15 at 23:36
  • Travis supports deployment with many services like Heroku and AWS as well as pushing to Git remotes and executing Bash scripts for deployment. I wanted to know how to push to a Git remote, but not a GitHub Git remote. I have a remote on my server. I've finally gotten it to work today! The challenging part was encrypting the SSH key for pushing to the remote through SSH and actually getting Travis to use the key. – IIllIIll Dec 25 '15 at 01:25
  • Why use `|` after `after_success:` in step 4? – nn0p Feb 27 '16 at 16:08
  • 3
    @nn0p that's just standard YAML multiline string notation: http://stackoverflow.com/questions/3790454/in-yaml-how-do-i-break-a-string-over-multiple-lines – Ciro Santilli OurBigBook.com Feb 27 '16 at 19:51
  • It is CRITICAL that you make sure your tokens don't leak. Note the `-q` flag passed to `git push`. `git push` and `git pull` often print the remote URL, which contains the GH token. I would go a step further and always pipe all output to `/dev/null` or store into a variable; I don't know what the behavior is of `git push/pull` if an error is encountered with the `-q` flag, it may barf your token to stdout or stderr if this happens.. – zbeekman Jun 13 '16 at 13:54
  • 2
    I think `https://:$GITHUB_API_KEY@github.com` may need to be `https://$GITHUB_API_KEY:@github.com` or `https://$GITHUB_API_KEY@github.com`. – Benjamin Oakes Oct 22 '16 at 02:23
  • I think you mean `2>/dev/null` rather than `&2>/dev/null`. – davidchambers Sep 25 '18 at 20:39
  • 1
    I had to change the URL to `https://:$GITHUB_API_KEY@github.com//.git` for it to work. – John Blischak Nov 09 '18 at 21:29
35

I don't know how recent it is, but Travis now have a built-in deployment option, basically add to your travis file :

deploy:
  provider: pages
  skip_cleanup: true
  local_dir: myfolder/  # or remove this line to upload from root of repo
  github_token: $GITHUB_TOKEN # Set in travis-ci.org dashboard
  on:
    branch: master

Make sure you don't have a .gitignore in the uploaded folder ; it only uploads non ignored files.

See the online official doc from travis : https://docs.travis-ci.com/user/deployment/pages/

There is no public key issue using "Repository Settings" approach, you generate a key in Github then copy paste it into secret/non visible fields of Travis.

Upload history issue : Note that each upload crushes any previously uploaded data, without preserving history.

  • You can now (Nov 2017+) instead preserve history by adding a keep_history: true line

  • This may be desirable as these snapshot builds can be voluminous, and they are reproducible at will anyway (simply branch your depot back from the revision you want). Pointing to such artifacts is typically pointing to a last successful build of a snapshot.

  • However to trigger storage to a stable place, simply edit your travis to add flag :
    target_branch: Branch to push force to, defaults to gh-pages
    E.g target_branch : rc1.2

And run it once before setting it back to snapshot mode.

Another alternative that might be good for releases (I haven't personally tested though) is to publish to a Tag see : https://docs.travis-ci.com/user/deployment/releases/

Yann TM
  • 1,942
  • 13
  • 22
  • 1
    Drawbacks: (1) Needs a _personal access token_ from GitHub, unnecessarily granting access to all repos of the user. (2) Overwrites the target repo branch without preserving git history. – Thomas Weiser Apr 20 '17 at 20:46
  • Concerning (1) it's not so much of a problem, travis is not going to mess with any other repos you have except for the one concerned by the build. The issue was making it public, it's private and shared only to travis, that you trust implicitly since it's making the upload. (2) is actually desirable, you don't want to store the history of binary results of the build. Just tag revisions for release. – Yann TM May 05 '17 at 20:44
  • There is a repo of example using this solution? – Manoel Vilela Feb 16 '19 at 16:02
  • @ManoelVilela here is a production example, that uploads to a different place (branch) the MacOS and linux targets. https://github.com/lip6/libDDD/blob/master/.travis.yml – Yann TM Feb 28 '19 at 22:40
12

The travis-ci documentation here recommends adding this to push to a git repo:

after_success:
   - chmod 600 .travis/deploy_key.pem # this key should have push access
   - ssh-add .travis/deploy_key.pem
   - git remote add deploy DEPLOY_REPO_URI_GOES_HERE
   - git push deploy

However, this is insecure as it has you store your unprotected private key in the github repository.

Instead you can add your ssh key as a encrypted environmental variable using the travis tool:

travis encrypt DEPLOY_KEY=<private ssh key with write access> --add env.matrix

Now you just need to add this line to the beginning of after_success:

cat $DEPLOY_KEY > .travis/deploy_key.pem

Please note that after_success will toggle in every build in the build matrix so if you have multiple jobs per build your code will get pushed multiple times, which won't do anything but is good to know that it is occurring.

Paul
  • 26,170
  • 12
  • 85
  • 119
joshua-anderson
  • 4,458
  • 5
  • 35
  • 56
  • i will try this out - please stand by for the star – Stasik Apr 26 '14 at 19:31
  • the key was too long for secured env variable, see my alternative solution – Stasik Apr 27 '14 at 10:00
  • 2
    I solved the problem of key too long using the option of encrypting the file: http://docs.travis-ci.com/user/encrypting-files/ – Elias Dorneles Sep 07 '15 at 15:08
  • This looks promising, but how in the world did you figure this out? Their docs are very... Barebones in all of the wrong ways... – IIllIIll Dec 24 '15 at 01:29
  • How do the deployment just on approved PR, which means on merge event. I saw the use of $TRAVIS_PULL_REQUEST but I dont want to deploy on each PR, just on approved PR. – Ron Jun 25 '19 at 14:50
10

Just to add another solution, I used a HTTPS token from github, encrypted it and used HTTPS for checkouts and pushes

Stasik
  • 2,568
  • 1
  • 25
  • 44
  • 1
    I would vote for this being a better solution. To make the token, go to Account settings -> Applications -> Generate Token. There is a nice guide here: https://medium.com/@nthgergo/publishing-gh-pages-with-travis-ci-53a8270e87db – Jonas Jongejan Nov 20 '14 at 05:01
  • 1
    Note that HTTPS tokens cannot be limited to one repo. But since the token is stored encrypted and you are the only one who should have access to Travis CI it should not matter. – lumbric May 14 '15 at 11:00
  • So, how does your final solution look like? – okutane Sep 13 '16 at 16:56
  • @okutane https://github.com/open62541/open62541/blob/0.2/tools/travis/travis_push_doc.sh – Stasik Sep 14 '16 at 19:15
0

I just wrote a blog about this some days ago. Here's the brief:

I wrote a custom deploy script for this purpose. The core functionality of the script looks like this:

#!/bin/bash

git clone --depth=1 --branch=master "https://github.com/iBug/iBug.github.io.git" deploy
cd deploy
git rm -rf .
cd ..
mv _site/* deploy
cd deploy
git add --all
git config user.name "Travis CI"
git config user.email "travis@travis-ci.org"
git commit --message "Auto deploy from Travis CI"
git remote add deploy "https://$GH_TOKEN@github.com/iBug/iBug.github.io.git" &>/dev/null
git push deploy master &>/dev/null

Now go to https://github.com/settings/tokens and generate a token. Grant it public_repo privilege. Go to repository settings on Travis CI and store the token with the variable name being GH_TOKEN.

Add the deploy script to travis:

script: bundle exec jekyll build
after_success:
    - bash .travis/deploy.sh

Push these things to GitHub and Travis will be triggered.

My blog is here. It's comprehensive and thus redundant if posted as an answer here (because Stack Overflow users are mostly experienced developers). The script I posted in my blog also lacks a functionality: It does not preserve commit history of the built site, whereas the script in this answer above does.

iBug
  • 35,554
  • 7
  • 89
  • 134