A Node app that runs in development but stops in production

Question

I am having trouble finding what's wrong with my code. This app runs fine if I run "node whatever.js" but once I push it up to production in a place like Nodejitsu, it fails to run (I'm actually not sure if it fails to run, but I think it just runs once right after I activate the server, and then stops running for all requests afterwards). (I deleted my keys and tokens information for privacy)

P.S: the actual app is hosted here: http://bucifer-realtime-twitter.jit.su/ The "wait for it" in the template is supposed to change into a whole bunch of twitter data using socket.io/Express, which works fine in development.

var express = require("express");
var fs = require("fs");
var http = require("http");
var https = require("https");
var port = "1337";
var twitter = require('ntwitter');

var twit = new twitter({
  consumer_key: 'x',
  consumer_secret: 'x',
  access_token_key: 'x',
  access_token_secret: 'x'
});

var app = express();
app.get("/", function(request, response) {
  var content = fs.readFileSync(__dirname + "/template.html");
  response.setHeader("Content-Type", "text/html");
  response.send(content);
});
var server = http.createServer(app);
server.listen(port);

var io = require('socket.io').listen(server);

app.use(express.static(__dirname + '/public'));

twit.stream('statuses/filter', { track: ['naruto'] }, function(stream) {
  stream.on('data', function (tweet) {
    console.log("working...");
    io.sockets.emit("tweet", tweet);
  });
  stream.on('end', function() {
    console.log("Disconnected");
    io.sockets.emit('ending', {message: 'test'});
  });
  stream.on('error', function(error ,code) {
    console.log("My error: " + error + ": " + code);
  });
  // Disconnect stream after -- seconds
  var myTimeOut = 21000;
  setTimeout(stream.destroy, myTimeOut);
});

@jimm101 in nodejitsu logs, I see nothing but the console messages that I coded in "working", "working" and "disconnected". And it says "Nodejitsu OK" meaning that the app worked fine. But the console messages are from way back when I first activated the app like 2 weeks ago and it does not fire everytime someone goes to the URL ... did I do something wrong that doesn't fire up the app everytime a request comes in? — Terry Bu, Aug 13 '14 at 18:14
What are you trying to do? twit.stream looks like it kills itself after 21 seconds? — jimm101, Aug 13 '14 at 18:38
@jimm101 wow you are so right. But I just wanted to make it so that for any request that comes in, twit stream runs and then stops after 21 seconds ... I think with this setup, it just stops running the whole app after 21 seconds ... do you have any ideas? (The reason for the timeout was that when I stream too much twit stream, Twitter puts a ban on my API Key so I wanted to stop twit stream from going out of control) — Terry Bu, Aug 13 '14 at 18:43
I think it just stops twit.stream(), but without that, the app won't do anything other than send template.html, and sit listening to an empty socket. Sounds like Twitter bans you for listening too much--they can't know you're streaming over socket.io. You can put a setInterval to wake up and check intermittently, and try to fly under their radar. Is the limit on how many things you download? If so, you could count by message/byte/whatever, and destroy/setTimeout(restart) again after enough time has elapsed to keep you out of trouble. — jimm101, Aug 13 '14 at 19:01

score 0 · Accepted Answer · answered Aug 13 '14 at 21:42

To summarize the comments above, in case anyone else has a similar situation ...

The lines

// Disconnect stream after -- seconds
var myTimeOut = 21000;
setTimeout(stream.destroy, myTimeOut);

disconnect the Twitter feed, so users are still served a page with an active socket.io connection, but there's nothing listening to Twitter to update them.

If Twitter is throttling the ability to listen, then the code quoted above needs to take a different approach, by either intermittently waking up to check messages, going dormant intermittently when bytes/message/whatever thresholds are nearing unacceptable levels, etc. Twitter will have no insight to the socket.io connection, so there's you don't need to limit that traffic for their sake.

A Node app that runs in development but stops in production

1 Answers1