0

I am looking for a recipe for writing and reading the raw data generated by a requests transaction from files rather than a socket. By "raw data" I mean the bytes just before they are written to or read from the underlying socket. I've tried:

  • Using "hooks". This seems to be mostly deprecated as the only remaining hook is "response".
  • mount()ing a custom Adapter. Some aggressive duck-typing here provides access to the underlying httplib.HTTPConnection objects, but the call stack down there is complicated and quite brittle.

The final solution does not need to be general-purpose as I am only interested in vanilla HTTP functionality. I won't be streaming or using the edgier parts of the protocol.

Thanks!

BrianTheLion
  • 2,618
  • 2
  • 29
  • 46
  • Are you interested in the data that the requests library sends to the server? Or the data that the server returns? Or both? And, what do you mean by "raw" data? What distinguishes raw data from non-raw data? – Robᵩ Aug 20 '14 at 04:01
  • I would think you could do this rather nicely with [httmock](https://pypi.python.org/pypi/httmock/) but I'm not in a place where I can start writing a proper answer. – tripleee Aug 20 '14 at 19:32
  • @tripleee Thanks for the pointer! I didn't know httmock existed. A look at the source code reveals that it does many of the things I tried with my "custom Adapter" solution, but better. Unfortunately it's still a long way from the actual socket. – BrianTheLion Aug 20 '14 at 19:52
  • For interactive examination, I'd use wireshark. – Robᵩ Aug 21 '14 at 04:19

1 Answers1

0

Spawn a thread (import threading). Run an HTTP server in there. You can generate a unique port on demand by socket.socket().bind(0). In the HTTP server, just write the incoming data to a file (perhaps named by timestamp and incoming port number). Then send your requests there.

John Zwinck
  • 239,568
  • 38
  • 324
  • 436