1

(This is a follow-up question to this one)

How can I tell urllib3 to log the FULL request, including, but not limited to:

  • URL
  • query parameters
  • headers
  • body
  • and anything else which is sent inside the request (I am not sure there is anything else, but if there is something else, I also want to see it)

I am having trouble connecting to LinkedIn with OAuth (a similar implementation works with Google and Facebook), and I would like to see exactly what requests are being sent. I suspect the auth_token is not being provided, but I need to confirm this. For that, I need urllib3 to show the full requests, since they are over HTTPS and I can not analyze network traffic to see them (end-to-end encryption).

Community
  • 1
  • 1
blueFast
  • 41,341
  • 63
  • 198
  • 344
  • You should spend some time exploring the Requests API; I'm noticing a lot of your questions you could answer yourself if you read the rauth and Requests documentation. For example, access to the `PreparedRequest` object from the `Response` object is fully discussed in Requests' documentation. – maxcountryman May 02 '13 at 16:10
  • 1
    Sure you are right. If I had the time to read the documentation of Requests, urrlib3 and rauth (and any other dependency), remember all those details, and apply them to my particular case, deciding how to solve my specific problem, I would have solved this isssue myself. Heck, I would never need Stackoverflow! I am not even interested in the requests content: I was just trying to get OAuth2 working with linkedin, which is, strangely enough, expecting oauth2_access_token instead of access_token, the default in rauth (which can not be overridden), and wants no bearer token in the header. – blueFast May 03 '13 at 07:34
  • 1
    And just to clarify. The way I see it, you are the expert in your library. I am not. I - and others - need your help (which you can give or not) I assume you need the help from others regarding other issues, and that you are not expert in all areas. So often you will ask stupid or obvious questions. At least I do. So yes, you can complain that I am asking stupid questions (this is not the silliest question that I have asked), but I already knew that. Somewhere, someplace is the answer to my question. I just do not know where, and this is (for me) the right place to ask. – blueFast May 03 '13 at 07:40

2 Answers2

2

You can access the PreparedRequest object that was actually sent after the fact of a request by using the request accessor, e.g. print dir(r.request).

maxcountryman
  • 1,562
  • 1
  • 24
  • 51
0

You can hack into the requests hooks system instead, then use that to track responses and their requests:

from requests import hooks


_orig_default_hooks = hooks.default_hooks


def my_default_hooks():
    hooks = _orig_default_hooks()
    hooks['response'].append(response_hook)
    return hooks


hooks.default_hooks = my_default_hooks


# requests.models is imported by the requests package, so we need to ensure it's reference
# to default_hooks is updated too.
import requests.models
requests.models.default_hooks = my_default_hooks


def response_hook(r, **kw):
    req = r.request
    print req.headers

The reqeuests.hooks.default_hooks() function is called for each and every request created, and by injecting your own response event hook you get to see each and every response received. Responses have a .request attribute, which is a PreparedRequest instance, on which you'll find the .headers and .body attributes for your inspection.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343