How does the Python BaseRequestHandler rfile.read() method work?

Question

I'm sending a text file with a string in a Python script via POST to my server:

fo = open('data.txt','a')
fo.write("hi, this is my testing data")
fo.close()

with open('data.txt', 'rb') as f:
    r = requests.post("http://XXX.XX.X.X", data = {'data.txt':f})
    f.close()

And receiving and handling it here in my server handler script, built off an example found online:

def do_POST(self):
    data = self.rfile.read(int(self.headers.getheader('Content-Length')))
    empty = [data]
    with open('processing.txt', 'wb') as file:
        for item in empty:
            file.write("%s\n" % item)
                
    file.close()
    self._set_headers()
    self.wfile.write("<html><body><h1>POST!</h1></body></html>")

My question is, how does:

self.rfile.read(int(self.headers.getheader('Content-Length')))

take the length of my data (an integer, # of bytes/characters) and read my file? I am confused how it knows what my data contains. What is going on behind the scenes with HTTP?

It outputs data.txt=hi%2C+this+is+my+testing+data

to my processing.txt, but I am expecting "hi this is my testing data"

I tried but failed to find documentation for what exactly rfile.read() does, and if simply finding that answers my question I'd appreciate it, and I could just delete this question.

score 2 · Accepted Answer · edited Aug 30 '23 at 09:56

2

Your client code snippet reads contents from the file data.txt and makes a POST request to your server with data structured as a key-value pair. The data sent to your server in this case is one key data.txt with the corresponding value being the contents of the file.

Your server code snippet reads the entire HTTP Request body and dumps it into a file. The key-value pair structured and sent from the client comes in a format that can be decoded by Python's built in library urlparse.

Here is a solution that could work:

def do_POST(self):
    length = int(self.headers.get('content-length'))
    field_data = self.rfile.read(length)
    fields = urlparse.parse_qs(field_data)

This snippet of code was shamefully borrowed from: https://stackoverflow.com/a/31363982/705471

If you'd like to extract the contents of your text file back, adding the following line to the above snippet could help:

data_file = fields["data.txt"]

To learn more about how such information is encoded for the purposes of HTTP, read more at: https://en.wikipedia.org/wiki/Percent-encoding

edited Aug 30 '23 at 09:56

VisualPlugin Rōblox

3
2

answered Jun 15 '18 at 18:00

Axe

6,285
3
31
38

I see, I think I understand. Could you clarify what 'rfile' is though? I understand read() is built into Python. – kasool Jun 15 '18 at 18:18
2

You can read more about this here: https://docs.python.org/2/library/basehttpserver.html#BaseHTTPServer.BaseHTTPRequestHandler.rfile. Basically, it is a stream containing all contents from the request body. This is usually setup by Python's built-in `BaseHTTPRequestHandler` or one of its derived classes for you to access when your function gets invoked. – Axe Jun 15 '18 at 18:26
2

why shameful? you should be unashamed to have properly credited the idea – WestCoastProjects Feb 03 '21 at 06:39
In Python 3, this will cause an `AttributeError: 'HTTPMessage' object has no attribute 'getheader'`. To fix, replace the `getheader()` call with `get()`. – Henrik Heimbuerger Nov 23 '22 at 11:31

How does the Python BaseRequestHandler rfile.read() method work?

1 Answers1

Linked