Creating a persistant connection to twitter stream API using R

Question

I am presently using the streamR package in R to stream tweets from the filter stream in twitter. I have a handshaken ROAuth object that I use for this. My piece of code looks like:

# load the Twitter auth object
load("twitter_oAuth3.RData")
load("keywords3.RData")

streamTweet = function(){
  require(streamR)
  require(ROAuth)

  stack = filterStream(file.name="",track=keywords,timeout=500,oauth=twitter_oAuth)
  return(stack)
}

I wanted to create a real time application, which involves dumping these tweets into an activeMQ topic. My code for that is:

require(Rjms)

# Set logger properties
url = "tcp://localhost:61616"
type = "T"
name = "TwitterStream"

# initialize logger
topicWriter = initialize.logger(url,type,name)

topicWrite = function(input){
#   print("writing to topic")
  to.logger(topicWriter,input,asString=TRUE,propertyName='StreamerID',propertyValue='1')
  return()
}

logToTopic = function(streamedStack){ 
#   print("inside stack-writer")
  stacklength = length(streamedStack)
  print(c("Length: ",stacklength))
  for(i in 1:stacklength){
    print(c("calling for: ",i))
    topicWrite(streamedStack[i])
    }
  return()
}

Now my problem is that of the timeout that filterStream() needs. I looked under the hood, and found this call that the function makes:

url <- "https://stream.twitter.com/1.1/statuses/filter.json"
    output <- tryCatch(oauth$OAuthRequest(URL = url, params = params, 
                                          method = "POST", customHeader = NULL, 
                                          writefunction = topicWrite, cainfo = system.file("CurlSSL", 
                                                                                             "cacert.pem", package = "RCurl")), error = function(e) e)

I tried removing the timeout component but it doesn't seem to work. Is there a way I can maintain a stream forever (until I kill it) which dumps each tweet as it comes into a topic?

P.S. I know of a java implementation that makes a call to the twitter4j API. I, however, have no idea how to do it in R.

score 0 · Answer 1 · edited Sep 23 '14 at 05:40

The documentation for streamR package mentions that the default option for timeout option in filterStream() is 0 which will keep the connection open permanently.

I quote:

"numeric, maximum length of time (in seconds) of connection to stream. The connection will be automatically closed after this period. For example, setting timeout to 10800 will keep the connection open for 3 hours. The default is 0, which will keep the connection open permanently."

Hope this helps.

Creating a persistant connection to twitter stream API using R

1 Answers1