3

On Twisted's website, they have an example called "Receiving Responses" which shows you how to get the response headers and response code among other things, but not the response body (the actual HTML body returned by the request to the website).

in def cbrequest(response), how would I print out the HTML text return by the GET request to example.com? They show methods like response.headers for getting headers, but I havent seen a method for returning the front end's body.

http://twistedmatrix.com/documents/12.1.0/web/howto/client.html#auto9

from pprint import pformat

from twisted.internet import reactor
from twisted.internet.defer import Deferred
from twisted.internet.protocol import Protocol
from twisted.web.client import Agent
from twisted.web.http_headers import Headers

class BeginningPrinter(Protocol):
    def __init__(self, finished):
        self.finished = finished
        self.remaining = 1024 * 10

    def dataReceived(self, bytes):
        if self.remaining:
            display = bytes[:self.remaining]
            print 'Some data received:'
        print display
        self.remaining -= len(display)

def connectionLost(self, reason):
    print 'Finished receiving body:', reason.getErrorMessage()
    self.finished.callback(None)

agent = Agent(reactor)
d = agent.request(
    'GET',
    'http://example.com/',
    Headers({'User-Agent': ['Twisted Web Client Example']}),
    None)

def cbRequest(response):
    print 'Response version:', response.version
    print 'Response code:', response.code
    print 'Response phrase:', response.phrase
    print 'Response headers:'
    print pformat(list(response.headers.getAllRawHeaders()))
    finished = Deferred()
    response.deliverBody(BeginningPrinter(finished))
    return finished
d.addCallback(cbRequest)

def cbShutdown(ignored):
    reactor.stop()
d.addBoth(cbShutdown)

reactor.run()

Edit: I tried printing response.deliverBody(BeginningPrinter(finished)) to get the response text, but to no avail

david attenbrough
  • 75
  • 1
  • 1
  • 7

1 Answers1

0

When I indent your connectionLost method so it's properly a method on BeginningPrinter and not just a free-floating function, this works fine. I did run it on Twisted 15.4.0, though, so perhaps you should try upgrading?

Glyph
  • 31,152
  • 11
  • 87
  • 129
  • That indentation error is just a product of pasting the text here and manually adding indents for it to show as a block of code. It's fine on my computer. I know how to print the response headers for example, but how do I print the response body? I want to do the equivalent of requests.get("http://www.example.com").text in the requests library. – david attenbrough Oct 05 '15 at 21:26