tweepy stops after a few hours

Question

I use tweepy for streaming some tweet. This is my procedure:

import tweepy
import json

consumer_key = "***" 
consumer_secret = "***"
access_token_key="***"
access_token_secret="***"

auth1 = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth1.set_access_token(access_token_key, access_token_secret)
api = tweepy.API(auth1)

class StreamListener(tweepy.StreamListener):
    def on_status(self, status):
        try:
            print status.text
        except Exception, e:
            print 'Encountered Exception Tweet:', e
            pass
        return True

    def on_error(self, status_code):
        print 'Encountered error with status code:' + repr(status_code)
        return True 

    def on_data(self, data):
        if 'in_reply_to_status_id' in data:
            status = tweepy.Status.parse(self.api, json.loads(data))
            if self.on_status(status) is False:
                return True
        elif 'delete' in data:
            delete = json.loads(data)['delete']['status']
            if self.on_delete(delete['id'], delete['user_id']) is False:
                return True
        elif 'limit' in data:
            if self.on_limit(json.loads(data)['limit']['track']) is False:
                return True
        return True

    def on_timeout(self):
        print 'Timeout...'
        return True

l = StreamListener()
streamer = tweepy.Stream(auth=auth1, listener=l, timeout=36000000)

setTerms = ['enbrel']
streamer.filter(follow=None,track = setTerms)

After two / three hours this procedure stops. No signal error, timeout, etc.. It just does not get more tweet. Where am I doing wrong?

Have you been able to reproduce this behaviour? It could have been a problem with your network connection, or Twitter's API having problems. — Aaron Hill, Mar 13 '14 at 18:48

Luigi · Answer 1 · 2014-04-27T11:54:11.010

9

Try adding an on_disconnect method to your class. It could be that Twitter is disconnecting you (not an error, also not a timeout) and you do not handle this. You can handle different Twitter errors differently, if you wish.

def on_disconnect(self, notice):
    """Called when twitter sends a disconnect notice

    Disconnect codes are listed here:
    https://dev.twitter.com/docs/streaming-apis/messages#Disconnect_messages_disconnect
    """
    return

Check out the streaming module of tweepy for more info.

You can also try to enable stall warnings in your streamer.filter(). Below are all the options and their default values from the Tweepy source:

def filter(self, follow=None, track=None, async=False, locations=None,
           stall_warnings=False, languages=None, encoding='utf8'):

edited Apr 27 '14 at 11:54

answered Apr 27 '14 at 11:48

Luigi

4,129
6
37
57

An old answer I know, but could you elaborate on this a bit? I want to reconnect on_disconnect, but I'm unsure of how to best do this. – Daniel Quinn Aug 01 '15 at 00:03
I figured it out: in my own loop, check for `stream.running`. If it isn't, delete the stream and start a new one after `n` seconds. – Daniel Quinn Aug 01 '15 at 12:04

score 1 · Answer 2 · edited Nov 10 '17 at 14:09

1

You may want to initiate the api with a time out to start with

api = tweepy.API(auth1,timeout=60)

edited Nov 10 '17 at 14:09

Luigi

4,129
6
37
57

answered Jul 15 '17 at 07:27

adomakor412

91
5

tweepy stops after a few hours

2 Answers2