Extract data from Get requests in python

Question

I have got a SimpleHTTPServer in a raspberry pi working with a python script that is executed in the same directory as the index.html web page. The code is the following:

#!/usr/bin/python

import SimpleHTTPServer
import SocketServer
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
import httplib2

PORT = 8080

Handler = SimpleHTTPServer.SimpleHTTPRequestHandler

httpd = SocketServer.TCPServer(("", PORT), Handler)

print "serving at port", PORT
httpd.serve_forever()

Once the web page is loaded, it starts to send different get requests with data that I need to read with my python script, but I don know how to do it.

This is an example of the get requests;

10.8.0.6 - - [27/Nov/2016 11:18:07] code 404, message File not found
10.8.0.6 - - [27/Nov/2016 11:18:07] "GET /ok.png HTTP/1.1" 404 -
10.8.0.6 - - [27/Nov/2016 11:18:07] code 404, message File not found
10.8.0.6 - - [27/Nov/2016 11:18:07] "GET /red.png HTTP/1.1" 404 -
10.8.0.6 - - [27/Nov/2016 11:18:07] code 404, message File not found
10.8.0.6 - - [27/Nov/2016 11:18:07] "GET /arduino/start/0.16388046142178503 HTTP/1.1" 404 -
10.8.0.6 - - [27/Nov/2016 11:18:07] code 404, message File not found
10.8.0.6 - - [27/Nov/2016 11:18:07] "GET /favicon.ico HTTP/1.1" 404 -
10.8.0.6 - - [27/Nov/2016 11:18:17] code 404, message File not found
10.8.0.6 - - [27/Nov/2016 11:18:17] "GET /arduino/update/0.6913944096802204 HTTP/1.1" 404 -

Don't worry about the 404 error, it appears because the web has different icons that I did not put in the directory yet because I am doing tests.

The reason because I want to do that it's because I am "translating" a web sever to control a solenoid valve and different sensors that I had working on Arduino Yun. In arduino, the code reads the get requests, and depending on what is the get request asking for it responses with an XML package with the sensor data or it acts on the valve.

On arduino, the code for "catching" the get request as a String is the following;

if (client) {  // got client?
        String request = client.readString();
        request.trim();

And then the program looks for a word in the string and depending on what word is found it responses with different information. An example;

if (request=="inicial") {

                        // send rest of HTTP header
                        client.println("Content-Type: text/xml");
                        client.println("Connection: keep-alive");
                        client.println();
                        // send XML file containing input states
                        inicial(client);
                    }

Summaraizing, I want to read the get requests as a String in my python script and extract data from them, and then response with an XML, but I think this last step will be better explained on the internet so I don't worry about it.

I hope my explanation was enough clear.

Thanks!!!

Are you trying to parse the webserver log messages? – Mohammad Yusuf Nov 27 '16 at 11:53 — Mohammad Yusuf, Nov 27 '16 at 11:53

Mohammad Yusuf · Answer 1 · 2016-11-27T15:34:21.013

0

This is how you can make request to an address. The response object is assigned to the variable. Then you can extract anything you want from it.

>>> r = requests.get('https://api.github.com/user', auth=('user', 'pass'))
>>> r.status_code
200
>>> r.headers['content-type']
'application/json; charset=utf8'
>>> r.encoding
'utf-8'
>>> r.text
u'{"type":"User"...'
>>> r.json()
{u'private_gists': 419, u'total_private_repos': 77, ...}

edited Nov 27 '16 at 15:34

answered Nov 27 '16 at 14:28

Mohammad Yusuf

16,554
10
50
78

Thanks for your answer but there must be a better way to do that. Isn't there any way to read the get requests directly, not just reading the log? – gcp900 Nov 27 '16 at 15:24
Actually I couldn't understand fully what you are trying to do. You can make a request and read the data using requests module in python. Can you tell me exactly what you are looking for? – Mohammad Yusuf Nov 27 '16 at 15:26
Simplehttpserver serves a web on port 8080. I introduce the raspberry address and port to access remotely the web from my browser. Once loaded the web, it makes get requests from my browser to the raspberry server to retrive data from a temperature sensor (for example), and then my python script needs to recognize the get request and sends back the data in XML format. Do you understand now? Thanks for your help. – gcp900 Nov 27 '16 at 17:03
It is as simple as communication between a python script on a raspberry pi (the server) and a web loaded on a remote web browser using simplehttpserver (the client) and using get requests to ask server and XML package to send back data to web. – gcp900 Nov 27 '16 at 17:09
Well you are mixing up many terms I'm still not clear. But I got a clue that you are trying to parse get parameters on server side. Read this: http://stackoverflow.com/questions/8928730/processing-http-get-input-parameter-on-server-side-in-python – Mohammad Yusuf Nov 27 '16 at 17:22
Use flask instead of simplehttpserver. https://www.raspberrypi.org/learning/python-web-server-with-flask/worksheet/ – Mohammad Yusuf Nov 27 '16 at 17:40
This is the most similar post I have found. It uses a conditional to check a parameter in a request as I did in arduino.http://stackoverflow.com/questions/2422055/how-to-check-if-request-get-var-is-none – gcp900 Nov 27 '16 at 17:44

Extract data from Get requests in python

1 Answers1