How do I receive Github Webhooks in Python

Question

Github offers to send Post-receive hooks to an URL of your choice when there's activity on your repo. I want to write a small Python command-line/background (i.e. no GUI or webapp) application running on my computer (later on a NAS), which continually listens for those incoming POST requests, and once a POST is received from Github, it processes the JSON information contained within. Processing the json as soon as I have it is no problem. The POST can come from a small number of IPs given by github; I plan/hope to specify a port on my computer where it should get sent.

The problem is, I don't know enough about web technologies to deal with the vast number of options you find when searching.. do I use Django, Requests, sockets,Flask, microframeworks...? I don't know what most of the terms involved mean, and most sound like they offer too much/are too big to solve my problem - I'm simply overwhelmed and don't know where to start.

Most tutorials about POST/GET I could find seem to be concerned with either sending or directly requesting data from a website, but not with continually listening for it.

I feel the problem is not really a difficult one, and will boil down to a couple of lines, once I know where to go/how to do it. Can anybody offer pointers/tutorials/examples/sample code?

Answer 1

First thing is, web is request-response based. So something will request your link, and you will respond accordingly. Your server application will be continuously listening on a port; that you don't have to worry about.

Here is the similar version in Flask (my micro framework of choice):

from flask import Flask, request
import json

app = Flask(__name__)

@app.route('/',methods=['POST'])
def foo():
   data = json.loads(request.data)
   print "New commit by: {}".format(data['commits'][0]['author']['name'])
   return "OK"

if __name__ == '__main__':
   app.run()

Here is a sample run, using the example from github:

Running the server (the above code is saved in sample.py):

burhan@lenux:~$ python sample.py 
 * Running on http://127.0.0.1:5000/

Here is a request to the server, basically what github will do:

burhan@lenux:~$ http POST http://127.0.0.1:5000 < sample.json
HTTP/1.0 200 OK
Content-Length: 2
Content-Type: text/html; charset=utf-8
Date: Sun, 27 Jan 2013 19:07:56 GMT
Server: Werkzeug/0.8.3 Python/2.7.3

OK # <-- this is the response the client gets

Here is the output at the server:

New commit by: Chris Wanstrath
127.0.0.1 - - [27/Jan/2013 22:07:56] "POST / HTTP/1.1" 200 -

Answer 2

Here's a basic web.py example for receiving data via POST and doing something with it (in this case, just printing it to stdout):

import web

urls = ('/.*', 'hooks')

app = web.application(urls, globals())

class hooks:
    def POST(self):
        data = web.data()
        print
        print 'DATA RECEIVED:'
        print data
        print
        return 'OK'

if __name__ == '__main__':
    app.run()

I POSTed some data to it using hurl.it (after forwarding 8080 on my router), and saw the following output:

$ python hooks.py 
http://0.0.0.0:8080/

DATA RECEIVED: 
test=thisisatest&test2=25

50.19.170.198:33407 - - [27/Jan/2013 10:18:37] "HTTP/1.1 POST /hooks" - 200 OK

You should be able to swap out the print statements for your JSON processing.

To specify the port number, call the script with an extra argument:

$ python hooks.py 1234 

Answer 3

I would use:

https://github.com/carlos-jenkins/python-github-webhooks

You can configure a web server to use it, or if you just need a process running there without a web server you can launch the integrated server:

python webhooks.py

This will allow you to do everything you said you need. It, nevertheless, requires a bit of setup in your repository and in your hooks.

Late to the party and shameless autopromotion, sorry.

Answer 4

If you are using Flask, here's a very minimal code to listen for webhooks:

from flask import Flask, request, Response

app = Flask(__name__)

@app.route('/webhook', methods=['POST'])
def respond():
    print(request.json) # Handle webhook request here
    return Response(status=200)

And the same example using Django:

from django.http import HttpResponse
from django.views.decorators.http import require_POST

@require_POST
def example(request):
    print(request.json) # Handle webhook request here
    return HttpResponse('Hello, world. This is the webhook response.')

If you need more information, here's a great tutorial on how to listen for webhooks with Python.

Answer 5

If you're looking to watch for changes in any repo...

1. If you own the repo that you want to watch

• In your repo page, Go to settings
• click webhooks, new webhook (top right)
• give it your ip/endpoint and setup everything to your liking
• use any server to get notified

2. Not your Repo

• take the url you want i.e https://github.com/fire17/gd-xo/

• add /commits/master.atom to the end such as:

  https://github.com/fire17/gd-xo/commits/master.atom

• Use any library you want to get that page's content, like:

• filter out the keys you want, for example the element

response = requests.get("https://github.com/fire17/gd-xo/commits/master.atom").text
response.split("<updated>")[1].split("</updated>")[0]
'2021-08-06T19:01:53Z'

• make a loop that checks this every so often and if this string has changed, then you can initiate a clone/pull request or do whatever you like