0

I'm a newbie to Tornado and I'm trying to use it to make an async HEAD HTTP request. How is that done with Tornado?

My question is inspired by this one: How do you send a HEAD HTTP request in Python 2?

Community
  • 1
  • 1
etayluz
  • 15,920
  • 23
  • 106
  • 151

2 Answers2

1

In Python 3.5+:

from tornado.httpclient import AsyncHTTPClient
from tornado.ioloop import IOLoop

async def fetch_head():
    response = await http_client.fetch("http://www.google.com/")
    print(response.headers)
    IOLoop.current().stop()

http_client = AsyncHTTPClient()
IOLoop.current().add_callback(fetch_head)
IOLoop.current().start()

In real code, don't call "stop" until all processing is complete and your program is ready to exit.

In older Python replace async / await with gen.coroutine and yield:

from tornado import gen

@gen.coroutine
def fetch_head():
    response = yield http_client.fetch("http://www.google.com/")
    print(response.headers)
    IOLoop.current().stop()
A. Jesse Jiryu Davis
  • 23,641
  • 4
  • 57
  • 70
  • What if I want to do this every second again and again? Each time a different URL? This is for a crawling application that brute force checks to see if pages exist. – etayluz Apr 10 '17 at 19:45
  • Follow the example for [Running in the background](http://www.tornadoweb.org/en/latest/guide/coroutines.html#running-in-the-background). You want to fetch the URL within a try/catch block, do something if there's an exception, and then sleep for a second and loop. – A. Jesse Jiryu Davis Apr 11 '17 at 02:24
1

Add method="HEAD" to your AsyncHTTPClient.fetch() call. 

response = await http_client.fetch("http://example.com", method="HEAD")
Ben Darnell
  • 21,844
  • 3
  • 29
  • 50