2

I'm trying to use the rtweet-package to download some tweets from a certain hashtag. I've used a guide from a place called OpenCodez, and I've run into problems

Using the "search_tweets" function of the rtweet-package, I'm not able to download more than 5 tweets, while the limit of rtweet should be around 18.000 tweets.

I don't get any errors, but the "Downloading"-graphic when running my script simply stops at 10% (when trying to download n=2000).

I've tried using the "retryonratelimit=TRUE" without luck. I've reset my script, tried different tutorials to establish a connection - which all work fine - up until I'm actually using the search_tweets-function.

So this is my code to connect to the API:

api_key <- "xxxx"
api_secret_key <- "xxxx"
access_token <- "xxxx"
access_token_secret <- "xxxx"

## authenticate via web browser
token <- create_token(
  app = "xxxx",
  consumer_key = api_key,
  consumer_secret = api_secret_key,
  access_token = access_token,
  access_secret = access_token_secret)

And this is my "scraper":

my_tweets = search_tweets("#vmd19", n=2000, lang='en')

The resulting data-frame is simply 5 columns, which is odd, when there should be at least a couple of hundred tweets under the hashtag. I've tried different queries (hashtags etc.), without luck. The download stops looking like this:

Downloading [===>-------------------------------------]  10%

I cannot figure out what I'm doing wrong. Hopefully, someone can help me troubleshoot this!

TheRecruit
  • 184
  • 9
  • Try testing with other keywords to verify Natalie's useful comment below - try "coronavirus" - if you do not get about 17,000 within a few minutes, then it's some other issue! – aiatay7n Apr 24 '20 at 21:47

1 Answers1

2

This issue was addressed here: https://github.com/ropensci/rtweet/issues/364

It looks like it's because of the window from which you can gather tweets (about the last week). If the number of tweets available from that time window is less than the n in your search_tweets function, it will cut out before reaching 100%. So if you ask for 100 tweets with a certain term, and that term was only tweeted 7 times in the last week, it will stop downloading at 7%.

Natalie Gallagher
  • 31
  • 4