1

I am using Python and Requests to request data from Twitter's streaming API. I would like to request data with a set of parameters, and after a period of time, change the request with a new set of parameters.

In the following simple, working example, I ask the Twitter streaming API for tweets with the keyword 'python.' After an hour, I ask the API for the keyword 'ruby.' However, to do this, I am creating a new requests object; I am not changing the original object.

import requests
import json
import time

USER = 'user'
PW = 'pw'

def make_request(keyword):
    r = requests.post('https://stream.twitter.com/1/statuses/filter.json',
    data={'track': keyword}, auth=(USER,PW))

    for line in r.iter_lines():
        if line:
            print json.loads(line)

        if time.time() > start_of_last_request + 3600:
            break

count = 0

keywords = ['python', 'ruby']

while count < 2:
    start_of_last_request = time.time()

    make_request(keywords[count])

    count = count + 1

Ultimately, I will need to create a new query every hour and I am worried that I will be creating too many connections.

My questions are: Is there a better way to change the request to the Twitter API? Am I actually creating multiple connections? If so, how do I avoid this? (There's a suggestion on how to close the previous connections, but the solution doesn't quite make sense to me.)

I would appreciate any help. Thank you!

Community
  • 1
  • 1
Johann
  • 15
  • 3

1 Answers1

1

As you linked, in order to close the connection after you are done with the request, set the keep-alive value to False instead of the default True:

r = requests.post('https://stream.twitter.com/1/statuses/filter.json',
               data={'track': keyword}, auth=(USER,PW),
               config={'keep_alive':False})

This will make sure that the connection is closed once you're done with it.

TankorSmash
  • 12,186
  • 6
  • 68
  • 106
  • Pleasure was all mine. I did have to go do some learning for myself, as afar as what `keep-alive` was, so we both benefitted – TankorSmash Jul 09 '12 at 04:38