Python - Accept POST (raw body) data

Question

I am completely new to Python. I am using GitLab which offers system hook feature wherein i can specify a URL and it will send event details in the form of JSON POST data. When i create a RequestBin URL and provide that URL in GitLab's system hook, then in case of any event such as project creation, it sends the event details and i can see the same in RequestBin as shown in the snapshot below.

Now, i want to fetch this JSON data in some variable so i can process it as per my need but i'm not sure how to do read that data.

I've seen some posts which explain how to read JSON data but as you can see below in the screenshot, the FORM/POST PARAMETERS is showing as None. It's the raw body that contains all the details (in JSON format):

I have tried reading the data using Java and it works with the code shown below:

String recv;
String recvbuff="";
BufferedReader buffread = new BufferedReader(new InputStreamReader(request.getInputStream()));

while ((recv = buffread.readLine()) != null)
recvbuff += recv;
buffread.close();

System.out.println(recvbuff);
out.println(recvbuff);

Looking for something similar in Python.

So if I understand correctly you are looking for a way to set up a simple Python server to process some JSON-POST requests? I would suggest CherryPy for that, it's pretty easy to set up some functions accepting calls with JSON http://docs.cherrypy.org/en/latest/basics.html#dealing-with-json — Ixio, Dec 20 '15 at 18:28
Is CherryPy an alternative to Apacher server? Also, the request is in JSON format but it's being sent as raw data so i'm not really sure whether what you've suggested will work. — Technext, Dec 20 '15 at 18:34
@AlastairMcCormack: How come there is no specific question? It's written in the subject itself i.e., to receive raw data from POST request. The problem is i'm not aware how to accept data in Python. I did it using Tomcat + JSP + jq + shell but that doesn't looks like a good approach so i wanted to try using Python. — Technext, Dec 21 '15 at 16:38
Python doesn't handle HTTP requests, pe se, frameworks/modules/libraries do. You've updated your question with some incomplete Flask code but not shown how you're testing it. Your question is too broad, too haphazard, too scattergun and too vague to able to write a good answer. As someone with a SO reputation as high as yours, you should be asking good questions already, but for the benefit of others and you, please see: http://stackoverflow.com/help/how-to-ask — Alastair McCormack, Dec 21 '15 at 17:25
Stick with one framework, like Flask, and get to understand it fully rather than jumping from one to the next. Use simple tools like `curl` to test it and try to understand the underlying technologies like HTTP and Python before moving on. I fear that you're trying to run before you can walk — Alastair McCormack, Dec 21 '15 at 17:36
I agree with what you've said but since i am new to it, i wanted to know what framework should i stick to and whether that would suffice what i'm looking for. Also, i'm not sure whether what Apache provides (for my need i.e., web server functionality. Looks like the framework will but not sure) will be take care by the frameworks such as Flask besides getting the content. Apologies for my ignorance. — Technext, Dec 21 '15 at 17:42
@AlastairMcCormack: Thanks for the inputs about my original post and apologies for the same. I hope my latest edit is better. — Technext, Dec 21 '15 at 17:59

score 0 · Answer 1 · answered Dec 20 '15 at 19:00

0

I would suggest using CherryPy. It's a neat Python library that allows you to build a simple webserver application, it's fits pretty nicely in your use case: it can easily accept JSON requests (http://docs.cherrypy.org/en/latest/basics.html#dealing-with-json).

If you write a file called myserver.py with the following code:

#!/usr/bin/python3
import cherrypy

class Root(object):
    @cherrypy.expose
    @cherrypy.tools.json_in()
    def index(self):
        data = cherrypy.request.json
        # You can manipulate here your json data as you wish
        print(data['name'])

if __name__ == '__main__':
    cherrypy.quickstart(Root(), '/')

You can simply launch the server with the command line:

python3 myserver.py

And test it with the following curl command:

curl -H "Content-Type: application/json" -POST http://127.0.0.1:8080 -d '{"name": "test", "path": "/"}'

You will then see test printed in your server log.

answered Dec 20 '15 at 19:00

Ixio

517
6
21

I tried your suggestion and _your_ curl command works but when i provide this URL to my application's web hook, the log doesn't show anything. As i mentioned in AlastairMcCormack's comment, i have successfully tested the web hook using Java but there too many tools involved and i wanted a _better_ way to do the same job. :( The response is definitely sent as JSON (as shown in screenshot) by the application but it's as raw data (again as shown in the screenshot). So, may be i have to read all the data that is sent and not just JSON body (which doesn't contain the actual output). – Technext Dec 21 '15 at 17:15
Well is your application running the webhook on the same server as your myserver.py file? Because 127.0.0.1:8080 is a local address and cannot be accessed from outside your local network. If you want to bind your server IP address just look at the second answer in https://stackoverflow.com/questions/115773/how-do-i-configure-the-ip-address-with-cherrypy – Ixio Dec 21 '15 at 17:48
Yes, the application is running on the same IP and i have tried using both public and private IP but to no avail. – Technext Dec 21 '15 at 17:49
There isn't even a line showing that there was some kind of request? What if you use ```nc -l 8080``` instead of myserver.py, do you see any sign of the request? If not (and curl works) it looks as if your webhook doesn't work, maybe your Java code gets it's information in another way? – Ixio Dec 21 '15 at 18:11
There will not be any such kind of request because i am not sending any request; it's the GitLab application that triggers the hook when any event happens in it such as project creation, project deletion etc. I have updated my post with the Java code that works. Hope that might give you some clue as to what i'm trying to convey. – Technext Dec 21 '15 at 18:17

Alastair McCormack · Answer 2 · 2015-12-21T17:33:15.083

0

Your Flask application doesn't return any data so you're not going to see anything returned. You need to return something like:

return "test data"

Your screenshot is only showing the request not the response. You sent no form encoded parameters, which is why it's showing "None".

The correct Content-type for JSON is: application/json

edited Dec 21 '15 at 17:33

answered Dec 20 '15 at 19:28

Alastair McCormack

26,573
8
77
100

I'm not sure what you meant when you mentioned about screenshot showing _only_ the response. I created a link in RequestBin and assigned it in the web hook of my application. Whenever any event is triggered such as project creation etc., it sends details regarding that event in the form of JSON and that's sent as raw data and it's that data which i want to accept in some variable so that i can process it. – Technext Dec 21 '15 at 16:45
Please re-read my answer. It says "only shows the **request** not the response". I don't how how RequestBin works but all the elements of the screenshot are contexts of a request **not a response** – Alastair McCormack Dec 21 '15 at 17:29

Python - Accept POST (raw body) data

2 Answers2