logging request/response python requests

Question

I have an API that talks to different APIs, most of them are exposed as restful web services. I am using python requests and I am using requests_toolbet to log all the requests/responses for debugging purposes. My code looks like:

def call_api(self, paramas):
    print('----------> calling API')
    url = self.base_url + params...
    headers = {'Authorization': 'Bearer ' + BEARER_TOKEN}
    resp = requests.get(url, headers=headers)
    print(dump.dump_all(resp).decode('utf-8'))
    return resp.json()

As you might imagine, every single request have a pretty similar code for logging:

    resp = requests.get(url, headers=headers)
    print(dump.dump_all(resp).decode('utf-8'))

I was wondering if there is a way that I can remove the same code using decorators or so. I am a newbie in python so any help is really appreciated.

David Culbreth · Accepted Answer · 2023-02-28T16:23:51.527

Edit: I wanted to explain why implementing this as a decorator isn't a great idea. On a fundamental level, a decorator is just a modification of a function's behavior with another function. However, typically the desired net result of said function should not be substantially modified, because that makes it harder to read and identify what's actually happening. In the ideal case, the reader should be able to basically ignore the decorator for the immediate concerns. For example, creating a decorator to log the runtime or success of a function's execution would not inherently modify the shape of the input, or its expected result.

In this particular case, it could be confusing to a future reader (and to your IDE's Language server) that the nominally returned result from the function is fundamentally different than the actual resulting value -- the returned value is a tuple[str, str] containing (url, headers), where as the actual result after passing through is some arbitrary value from a JSON decode. I mean, sure you can indicate this type on the function's result type, but that's still directly contradicting what would be returned in the function itself.

There is a way to do this with decorators that could shorten your code significantly, but I'm not sure that decorators are the way to go in this case. In Python, it is much preferred to write explicit code that clearly represents what is happening, and doesn't obfuscate what's actually happening. Here's a better (in my opinion) way to approach this problem: make it its own function, like so...

import requests, dump

def make_request(url, headers):
  resp = requests.get(url, headers=headers)
  print(dump.dump_all(resp).decode('utf-8'))
  return resp.json()

class MyAPIAdapter:
  base_url = 'https://www.example.com/'
  
  def call_api(self, params):
    url = self.base_url + params...
    headers = {'Authorization': 'Bearer ' + BEARER_TOKEN}
    return make_request(url, headers)

This way, we can clearly see that the call_api execution does actually call out to something else, and other code is being executed,

Of course, there technically is a way to do this with a decorator, but it ends up being more lines of code, and kind of confusing, hence the preferred method above.

import requests, dump

def basic_request(func):
  def wrapper(*args, **kwargs):
    print('Calling %s...' % func.__name__)
    url, headers = func(*args, **kwargs)
    resp = requests.get(url, headers=headers)
    print(dump.dump_all(resp).decode('utf-8'))
    return resp.json()

  return wrapper

class MyAPIAdapter:
  base_url = 'https://www.example.com/'

  @basic_request
  def call_api(self, params):
    url = self.base_url + params...
    headers = {'Authorization': 'Bearer ' + BEARER_TOKEN}
    return (url, headers)

score 0 · Answer 2 · answered Jan 20 '20 at 14:49

Yes, you can use decorators. You can write your own for simple cases. Also try functool wraps.

An example from an article well explained

>>> import functools
>>> def echo(fn):
...     @functools.wraps(fn)
...     def wrapped(*v, **k):
...         ....
...    return wrapped
...
>>> @echo
>>> def f(x):
...    " I'm f, don't mess with me! "
...    pass
>>> f.__name__
'f'
>>> f.func_doc
" I'm f, don't mess with me! "
>>> f(('spam', 'spam', 'spam!'))
f(('spam', 'spam', 'spam!'))

One more related answer: What does functools.wraps do?

logging request/response python requests

2 Answers2