6

I am setting up continuous integration using Jenkins and GitHub Enterprise.

So far I have been able to easily create Jenkins projects and run jobs regularly.

  • Jenkins is using SSH public/private keys to authenticate and clone GitHub repos.
  • Jenkins runs on a Linux server that is inside my company's VPN network.

My next step is to configure Jenkins/GitHub so that each git push to GitHub would trigger a Jenkins build job. I have set up the post-commit webhook on GitHub, and enabled the GitHub hook trigger for GITScm polling checkbox in my Jenkins projects.

However, GitHub fails to connect to Jenkins.

Here is the log:

We couldn’t deliver this payload: failed to connect

{
  "zen": "Design for failure.",
  "hook_id": 287451636,
  "hook": {
    "type": "Repository",
    "id": 287451636,
    "name": "web",
    "active": true,
    "events": [
      "push"
    ],
    "config": {
      "content_type": "json",
      "insecure_ssl": "0",
      "url": "http://1.2.3.4/jenkins/github-webhook/"
    },
    "updated_at": "2021-03-19T07:45:12Z",
    "created_at": "2021-03-19T07:45:12Z",
    "url": "https://api.github.com/repos/mycompany/my_repo/hooks/287451636",
    "test_url": "https://api.github.com/repos/mycompany/my_repo/hooks/287451636/test",
    "ping_url": "https://api.github.com/repos/mycompany/my_repo/hooks/287451636/pings",
    "last_response": {
      "code": null,
      "status": "unused",
      "message": null
    }
  },
  "repository": {
    "id": 347030450,
    "node_id": "MDEwOlJlcG9zaXRvcnkzNDcwMzA0NTA=",
    "name": "my_repo",
    "full_name": "mycompany/my_repo",
    "private": true,
    "owner": {
      "login": "mycompany",
      "id": 11980925,
      "node_id": "MDEyOk9yZ2FuaXphdGlvbjExOTgwOTI1",
      "avatar_url": "https://avatars.githubusercontent.com/u/11980925?v=4",
      "gravatar_id": "",
      "url": "https://api.github.com/users/mycompany",
      "html_url": "https://github.com/mycompany",
      "followers_url": "https://api.github.com/users/mycompany/followers",
      "following_url": "https://api.github.com/users/mycompany/following{/other_user}",
      "gists_url": "https://api.github.com/users/mycompany/gists{/gist_id}",
      "starred_url": "https://api.github.com/users/mycompany/starred{/owner}{/repo}",
      "subscriptions_url": "https://api.github.com/users/mycompany/subscriptions",
      "organizations_url": "https://api.github.com/users/mycompany/orgs",
      "repos_url": "https://api.github.com/users/mycompany/repos",
      "events_url": "https://api.github.com/users/mycompany/events{/privacy}",
      "received_events_url": "https://api.github.com/users/mycompany/received_events",
      "type": "Organization",
      "site_admin": false
    },
    "html_url": "https://github.com/mycompany/my_repo",
    "description": "My test repo.",
    "fork": false,
    "url": "https://api.github.com/repos/mycompany/my_repo",
    "forks_url": "https://api.github.com/repos/mycompany/my_repo/forks",
    "keys_url": "https://api.github.com/repos/mycompany/my_repo/keys{/key_id}",
    "collaborators_url": "https://api.github.com/repos/mycompany/my_repo/collaborators{/collaborator}",
    "teams_url": "https://api.github.com/repos/mycompany/my_repo/teams",
    "hooks_url": "https://api.github.com/repos/mycompany/my_repo/hooks",
    "issue_events_url": "https://api.github.com/repos/mycompany/my_repo/issues/events{/number}",
    "events_url": "https://api.github.com/repos/mycompany/my_repo/events",
    "assignees_url": "https://api.github.com/repos/mycompany/my_repo/assignees{/user}",
    "branches_url": "https://api.github.com/repos/mycompany/my_repo/branches{/branch}",
    "tags_url": "https://api.github.com/repos/mycompany/my_repo/tags",
    "blobs_url": "https://api.github.com/repos/mycompany/my_repo/git/blobs{/sha}",
    "git_tags_url": "https://api.github.com/repos/mycompany/my_repo/git/tags{/sha}",
    "git_refs_url": "https://api.github.com/repos/mycompany/my_repo/git/refs{/sha}",
    "trees_url": "https://api.github.com/repos/mycompany/my_repo/git/trees{/sha}",
    "statuses_url": "https://api.github.com/repos/mycompany/my_repo/statuses/{sha}",
    "languages_url": "https://api.github.com/repos/mycompany/my_repo/languages",
    "stargazers_url": "https://api.github.com/repos/mycompany/my_repo/stargazers",
    "contributors_url": "https://api.github.com/repos/mycompany/my_repo/contributors",
    "subscribers_url": "https://api.github.com/repos/mycompany/my_repo/subscribers",
    "subscription_url": "https://api.github.com/repos/mycompany/my_repo/subscription",
    "commits_url": "https://api.github.com/repos/mycompany/my_repo/commits{/sha}",
    "git_commits_url": "https://api.github.com/repos/mycompany/my_repo/git/commits{/sha}",
    "comments_url": "https://api.github.com/repos/mycompany/my_repo/comments{/number}",
    "issue_comment_url": "https://api.github.com/repos/mycompany/my_repo/issues/comments{/number}",
    "contents_url": "https://api.github.com/repos/mycompany/my_repo/contents/{+path}",
    "compare_url": "https://api.github.com/repos/mycompany/my_repo/compare/{base}...{head}",
    "merges_url": "https://api.github.com/repos/mycompany/my_repo/merges",
    "archive_url": "https://api.github.com/repos/mycompany/my_repo/{archive_format}{/ref}",
    "downloads_url": "https://api.github.com/repos/mycompany/my_repo/downloads",
    "issues_url": "https://api.github.com/repos/mycompany/my_repo/issues{/number}",
    "pulls_url": "https://api.github.com/repos/mycompany/my_repo/pulls{/number}",
    "milestones_url": "https://api.github.com/repos/mycompany/my_repo/milestones{/number}",
    "notifications_url": "https://api.github.com/repos/mycompany/my_repo/notifications{?since,all,participating}",
    "labels_url": "https://api.github.com/repos/mycompany/my_repo/labels{/name}",
    "releases_url": "https://api.github.com/repos/mycompany/my_repo/releases{/id}",
    "deployments_url": "https://api.github.com/repos/mycompany/my_repo/deployments",
    "created_at": "2021-03-12T10:40:44Z",
    "updated_at": "2021-03-18T13:59:52Z",
    "pushed_at": "2021-03-18T13:59:49Z",
    "git_url": "git://github.com/mycompany/my_repo.git",
    "ssh_url": "git@github.com:mycompany/my_repo.git",
    "clone_url": "https://github.com/mycompany/my_repo.git",
    "svn_url": "https://github.com/mycompany/my_repo",
    "homepage": null,
    "size": 3,
    "stargazers_count": 0,
    "watchers_count": 0,
    "language": "Makefile",
    "has_issues": true,
    "has_projects": true,
    "has_downloads": true,
    "has_wiki": true,
    "has_pages": false,
    "forks_count": 0,
    "mirror_url": null,
    "archived": false,
    "disabled": false,
    "open_issues_count": 0,
    "license": null,
    "forks": 0,
    "open_issues": 0,
    "watchers": 0,
    "default_branch": "main"
  },
  "sender": {
    "login": "MyLogin",
    "id": 63738097,
    "node_id": "MDQ6VXNlcjYzNzM4MDk3",
    "avatar_url": "https://avatars.githubusercontent.com/u/13818097?v=4",
    "gravatar_id": "",
    "url": "https://api.github.com/users/MyLogin",
    "html_url": "https://github.com/MyLogin",
    "followers_url": "https://api.github.com/users/MyLogin/followers",
    "following_url": "https://api.github.com/users/MyLogin/following{/other_user}",
    "gists_url": "https://api.github.com/users/MyLogin/gists{/gist_id}",
    "starred_url": "https://api.github.com/users/MyLogin/starred{/owner}{/repo}",
    "subscriptions_url": "https://api.github.com/users/MyLogin/subscriptions",
    "organizations_url": "https://api.github.com/users/MyLogin/orgs",
    "repos_url": "https://api.github.com/users/MyLogin/repos",
    "events_url": "https://api.github.com/users/MyLogin/events{/privacy}",
    "received_events_url": "https://api.github.com/users/MyLogin/received_events",
    "type": "User",
    "site_admin": false
  }
}

What could be the problem?

Danijel
  • 8,198
  • 18
  • 69
  • 133

2 Answers2

3

Well the problem is exactly because github cannot access your internal network.

There's an article in Jenkins Community Blog where author suggests using smee (released under ISC License) for webhooks forwarding.

Sergey
  • 140
  • 9
0

If using smee to deliver webhooks from GitHub to your VPNed Jenkins (you set up Github to send events to smee.io, and than you use the Smee client to retrieve the events and use them internally on Jenknins), here is a quick way to set it up.

  1. Go to Smee.io and get yourself a "channel", write down the URL you get (for example https://smee.io/Wua8D46nBvJ63nPP).

  2. Install Smee on your Jenkins: sudo npm install --global smee-client

  3. Configure the internal forwarding: smee -u https://smee.io/Wua8D46nBvJ63nPP --path /github-webhook/ --port 8080

  4. Go to your GitHub repository: Settings->Webhooks->Add webhook, and paste the URL under "Payload URL", choose Content type "application/json".

  5. Setup your Smee to run as a service on Jenkins, to start at boot time. You will need two files:

  • Init script /etc/init.d/smee
  • systemctl service description file /etc/systemd/system/smee.service

/etc/init.d/smee:

#! /bin/sh
### BEGIN INIT INFO
# Provides:          smee
# Required-Start:    $all
# Required-Stop:     
# Default-Start:     2 3 4 5
# Default-Stop:      
# Short-Description: Start SMEE daemon at boot time
# Description:       Forwards GitHub webhook notifications to Jenkins.
### END INIT INFO

PATH=/bin:/usr/bin:/sbin:/usr/sbin
DAEMON=/usr/bin/smee
PIDFILE=/var/run/smee.pid

test -x $DAEMON || exit 0

. /lib/lsb/init-functions

case "$1" in
  start)
 log_daemon_msg "Starting smee" "smee"
 start_daemon -p $PIDFILE $DAEMON
 log_end_msg $?
    ;;
  stop)
 log_daemon_msg "Stopping smee" "smee"
 killproc -p $PIDFILE $DAEMON
 log_end_msg $?
    ;;  
  force-reload|restart)
    $0 stop
    $0 start
    ;;
  status)
    status_of_proc -p $PIDFILE $DAEMON atd && exit 0 || exit $?
    ;;
  *)
    echo "Usage: /etc/init.d/smee {start|stop|restart|force-reload|status}"
    exit 1
    ;;
esac

exit 0

/etc/systemd/system/smee.service:

[Unit]
Description=smee.io webhook delivery from GitHub
After=network.target
StartLimitIntervalSec=0

[Service]
Type=simple
Restart=always
RestartSec=1
User=jenkins
ExecStart=/usr/bin/smee -u https://smee.io/Wua8D46nBvJ63nPP --path /jenkins/github-webhook/ --port 8080

[Install]
WantedBy=multi-user.target

Start the service: sudo /etc/init.d/smee start

Check the status: sudo /etc/init.d/smee status

The notifications from your repository should now be sent to Jenkins and Jenkins should be able to see and handle them. To quickly check if Jenkins handles any Smee comands, open the "Dashboard->System Log" and search for "smee".

Danijel
  • 8,198
  • 18
  • 69
  • 133