3

I'm trying to download tweets using twitteR package in R. I have a list of twitter users and I want to download their tweets. I keep getting the error

Error in .self$twFromJSON(out) : 
  Error: Rate limit exceeded. Clients may not make more than 150 requests per hour.

How can I slow down the request so it can keep downloading within the limits and without interruption? Below is a sample code.

# Load package
library(twitteR)

# Read list of users from file
users <- read.table('listOfTwitterUsers.txt')

# Pause for 10 sec
sleepTime = 10

for (user in users){

   # Download latest 2000 tweets from the user's timeline
   tweets <- userTimeline(user, 2000)

   # Extract tweets
   tweets <- unlist( lapply(tweets, function(t) t$getText() ) )

   # Save tweets to file
   write.csv(tweets, file=paste("Downloads/", user, ".csv", sep=""), row.names=F)
   Sys.sleep(sleepTime)
}
maiaini
  • 692
  • 1
  • 9
  • 13
  • 4
    some example codes that you are using would have been great.If you are using some kind of a loop, you can check out `Sys.sleep` to pause between the loops so that it is within the download limits – sayan dasgupta Feb 23 '12 at 12:09
  • This question http://stackoverflow.com/questions/9192698/lookup-twitter-followers-in-r/9193556#9193556 shares some similarities. – Paul Hiemstra Feb 23 '12 at 12:10
  • Take a look at Twitter's [Streaming API docs](https://dev.twitter.com/docs/streaming-api/methods#follow) and [this question](http://stackoverflow.com/questions/9253414/twitter-error-401-accessing-1-statuses-sample-json-reason-unauthorized). – jbaums Feb 23 '12 at 12:26
  • 1
    Thank you Sayan! Sys.sleep is the way to go. Would it possible to optimize how long to pause between requests? – maiaini Feb 23 '12 at 13:50
  • 1
    My guess at an optimal sleep time would be 3600/150 = 24 sec. If the request itself, and writing to csv, is slow, then you could `system.time` that, and cut off some of the sleep time to compensate. – jbaums Feb 23 '12 at 14:14
  • You can increase your API/limit to 350 requests/hour if you authenticate with a developer key/secret. You can create an app at dev.twitter.com – Btibert3 May 03 '12 at 16:42

1 Answers1

0

According to twitter API documentation the rate limit is 180 or 300 requests every 15 minutes, depending on the authorization. Therefore you should adjust your sys.sleep to this number.

https://dev.twitter.com/rest/reference/get/statuses/user_timeline

Justus Niemzok
  • 63
  • 5