4

I need to use the Twitter Stream API to stream tweet data to my firebase cloud function like this:

client.stream('statuses/filter', params,  stream => {

  stream.on('data', tweet => {
    console.log(tweet);
  })

  stream.on('error', error => {
    console.log(error)
  })

})

The stream is continuous but the firebase cloud function shuts down after a certain period of time. What solution could I make use of to be able to continuously receive the stream data?

Brendan
  • 834
  • 1
  • 9
  • 20

2 Answers2

5

Cloud Functions have a max running time of 540 seconds as documented. You would have to look at probably using a Compute Engine Instance from Google Cloud where you can have code running without limits. Or you could look at using the Google Cloud Scheduler to run your function every x time to get new tweets.

Jack Woodward
  • 991
  • 5
  • 9
  • Thank you, this was helpful. I think I will either use a Compute Engine Instance or use another platform such as Heroku to run the backend. – Brendan Nov 16 '18 at 10:01
1

The accepted response suggests running GCE and while it's certainly correct, I'd like to point out that anyone who was interested in Cloud Functions - a serverless solution - might find GAE (App Engine) much more viable for streaming data.

Our application utilises App Engine Standard as an ingestion service and it works like a charm - removing overhead required by GCE. If advanced networking features are required by your app App Engine Flexible or GKE (Kubernetes Engine) might also be something to look at!

user2997781
  • 11
  • 3