3

I am new to Python and Tweepy and I have been playing around with it for a while. I found this piece of code by gumption. I have been using it to fill up an Excel-File with tweets, which works well.

But I have a couple of questions: how many tweets can I get using this loop? Or to be more specific, when I get the first 2000 for the last few days, are those all the tweets posted for the tag I searched for? Or is it the same as with the Streaming API, that I can get 1% of all tweets?

And I don't really understand my rate limits. How many queries does it take to get 10.000 tweets using the aforementioned code? Is it 100 tweets per page, so 100 pages needed = 100 queries?

Community
  • 1
  • 1
Moses243
  • 33
  • 1
  • 4

1 Answers1

9

When I get the first 2000 for the last days, are those all the tweets posted for the tag I searched for? Or is it the same as with the Stream API, that I can get 1% of all tweets?

Neither. Twitter makes tweets available up to two weeks in the past. However, this isn't the same for all keywords:

The Search API is not complete index of all Tweets, but instead an index of recent Tweets. At the moment that index includes between 6-9 days of Tweets.

Additionally, it is not guaranteed that all recent tweets are indexed by the Twitter search API:

It’s important to know that the Search API is focused on relevance and not completeness. This means that some Tweets and users may be missing from search results.

This information comes from the Twitter Search API's general information page, which is a good read and will be able to answer a lot of questions.

And I don't really understand my rate limits. How many queries does it take to get 10.000 tweets using the aforementioned code ?

The Twitter API has a chart explaining rate limits for different resources. Rate limits come in 15-minute chunks. For Search, you have 180 requests per 15-minute window. Since you can get 100-count per request, in theory you can collect 180*100 = 18k tweets per 15 minutes. However, you can only collect as many tweets as Twitter makes available to you (see the 6-9 day inclusion). The number of tweets you can collect depends on your keywords and what Twitter makes available.

Luigi
  • 4,129
  • 6
  • 37
  • 57