0

I am trying this code to but it shows me error this is not my written code get from Avoid twitter api limitation with Tweepy

as i am new here i can't comment on that post please help me to solve this issue

auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(key, secret)

api = tweepy.API(auth, wait_on_rate_limit=True)
search = "#hashTag"

for tweet in tweepy.Cursor(api.search,
                           q=search,
                           include_entities=True).items():
    while True: 
        try:
            #tweet = cursor.next()
            cursor.execute('''INSERT INTO MyTable(name, geo, image, source, timestamp, text, rt) VALUES(?,?,?,?,?,?,?)''',(tweet.user.screen_name, str(tweet.geo), tweet.user.profile_image_url, tweet.source, tweet.created_at, tweet.text, tweet.retweet_count))
        except tweepy.TweepError:           
            time.sleep(60 * 15)
            continue
        except StopIteration:
            break
        break
db.commit() 
db.close()

this error i got

tweepy.error.TweepError: Failed to send request: HTTPSConnectionPool(host='api.twitter.com', port=443): Read timed out. (read timeout=60)
Community
  • 1
  • 1
faysal
  • 31
  • 9
  • @4m1nh4j1 bro thats your code originally can you help me to fix this? – faysal May 07 '16 at 15:14
  • @aaron-hill can you help me in this code? – faysal May 09 '16 at 01:10
  • remove your db code. Test your Twitter API first. When it works then work on your DB work. – Ash May 09 '16 at 01:27
  • @Afshin it shows output in console but i want 20k-40k tweets data which is not possible in console i tried csv file also but after 3k tweets i got limitation error 403, then i saw this code so i tried sqlite3 but it gives that error :( can you help me this data mining script is a part of my university project – faysal May 09 '16 at 16:07
  • Why don't you write it in a file? open a file outside of the loop and write the output into the file. Take a look at this question http://stackoverflow.com/questions/21976599/tweepy-search-api-writing-to-file-error – Ash May 10 '16 at 03:57
  • @Afshin thanks bro i tried but still it gives me 2000 tweets after that it gives me twitter error but i need 20-30k tweets for that i am trying wait_on_rate_limit=True but its not working. My objective is to prove we can get 20-40k data from twitter without firehose – faysal May 10 '16 at 06:04
  • Do you wait enough? because after hitting a rate limit the program stops until a specific time elapses and then it starts again. Another reason might be that there might be no more than 2000 tweets on a specific hashtag. – Ash May 10 '16 at 06:17
  • @Afshin it gives me rate limit error 429 and then stop, and i check that hashtag have more then 32k posted tweets in one day even though i tried #mothersDay which has 125k+ tweets but no success :( – faysal May 10 '16 at 06:47
  • 1
    Don't use search use streaming with tracking keywords. That's how people stream hashtags. look at this sample: https://github.com/tweepy/tweepy/blob/master/examples/streaming.py. This will certainly work for you. Though I think the time has passed, you'll still get a lot of tweets on the topic. – Ash May 10 '16 at 07:11
  • @Afshin no i want that using REST search, the link i posted in my question i think they made that script working using REST search but i don't know how they did :/ – faysal May 10 '16 at 09:12

0 Answers0