Python requests: figuring out request full size

Question

How to get full size of request sent with Python requests (I'm not asking for response)
Should I simply add length of body (if apply) and headers?
Is something more what need to be considerate regarding request elements?

This apply for any request sent (GET, POST, DELETE) I wish to compute its size.

if it is a POST request, then you can get the length of request from the headers — Stack, Jun 18 '18 at 11:57
by "full size" are you talking about the length of the received HTTP? — Nazim Kerimbekov, Jun 18 '18 at 12:00
len() of what? Headers, body (if apply)? Something more should be taken under consideration like options of url? — pbaranski, Jun 19 '18 at 07:38
@pbaranski check this https://stackoverflow.com/a/23816211/8150371 — Stack, Jun 19 '18 at 10:30

score 4 · Answer 1 · edited Oct 07 '21 at 08:46

Short answer - size of request is sum of mainly size of method, URL, headers and body (if apply) so in approximation it will be:

import requests

response = requests.get('https://httpbin.org/get')

method_len = len(response.request.method)
url_len = len(response.request.url)
headers_len = len('\r\n'.join('{}{}'.format(k, v) for k, v in response.request.headers.items()))
body_len = len(response.request.body if response.request.body else [])

print(f'Request size {method_len + url_len + headers_len + body_len}')

If anything more apply please post comment.

Resources:

RFC documentation https://www.rfc-editor.org/rfc/rfc7230#section-3

request library repository member response: https://github.com/requests/requests/issues/4694

@Stack comment to this question where he mention Python requests - print entire http request (raw)?

Question about URI: Is URL a part of request size?

score 1 · Answer 2 · edited Sep 18 '21 at 11:02

Adding to @pbaranski's answer.

This would work for httpx. Recording both request and response data usage:

async def measure_data_usage(self, res: httpx.Response):
    # get size of req
    method_len = len(res.request.method)
    url_len = len('\r\n'.join(f'{value}' for value in res.request.url.raw))
    headers_len = len('\r\n'.join(f'{key}{value}'
                                  for key, value in
                                  res.request.headers.items()))
    body_len = len(res.request.content if res.request.content else ())
    total_req_size = method_len + url_len + headers_len + body_len

    # get size of resp
    url_len = len('\r\n'.join(f'{value}' for value in res.request.url.raw))
    headers_len = len('\r\n'.join(f'{key}{value}'
                                  for key, value in
                                  res.headers.items()))
    body_len = len(res.request.content if res.content else ())
    total_resp_size = url_len + headers_len + body_len

There's repetition. You can feel free to optimize according to your program. I'm leaving it in for clarity.

Python requests: figuring out request full size

2 Answers2