stack exceed on recursive function for framing tcp stream

Question

I have a function that is recursive in framing tcp stream. But when I reach around 1000+ packets received per second I suddenly get Max call stack exceeded or something along those lines.

My code is:

  var expectedRemaining = (this.expectedSz - this.receivedSz);

  if (buff.length > expectedRemaining) {
    var tmp = buff.slice(0, expectedRemaining);
    buff = buff.slice(expectedRemaining);

    recurse = true;
    this.inPacket.push(tmp);
    this.receivedSz = this.expectedSz;
  } else {
    this.inPacket.push(buff);
    this.receivedSz += buff.length;
  }

  if (this.receivedSz === this.expectedSz) {
    this.emit('data', Buffer.concat( this.inPacket, this.expectedSz));
    this.reset();
  }

  if (recurse) this.handleData(buff);

Any suggestions?

Instead of making this recursive, you could use a `setImmediate` call to allow the stack frame to pop and events to fire before coming back to your `handleData` function. — Austin Mullins, Feb 13 '15 at 23:21
@AustinMullins how would that be? Can I just wrap the calling of `handleData` in a `setImmediate` callback? — majidarif, Feb 13 '15 at 23:21

Austin Mullins · Answer 1 · 2015-02-14T02:40:38.173

1

Right now, your function looks something like this (psuedocode):

this.handleData = function handleData(buf) {
   if ( someCondition ) {
      // do stuff
      recurse = true;
   }
   else {
      // do other stuff
   }

   if (recurse) {
       this.handleData(buf);
   }
};

What I'm suggesting is that you implement the recursive behavior with a setImmediate call. This will allow the stack frame to be cleared and the data event to be emitted before entering your function again.

this.handleData = function handleData(buf) {
   if ( someCondition ) {
      // do stuff
      recurse = true;
   }
   else {
      // do other stuff
   }

   if (recurse) {
       setImmediate(function() {
           this.handleData(buf);
       });
   }
};

Just in case people don't read the comments, for the application described in the original question, nextTick ended up being a better fit. The main difference is that nextTick guarantees that the given function will execute before any queued events.

edited Feb 14 '15 at 02:40

answered Feb 13 '15 at 23:29

Austin Mullins

7,307
2
33
48

h/t to the docs: http://nodejs.org/api/timers.html#timers_setimmediate_callback_arg – Austin Mullins Feb 13 '15 at 23:29
1

something similar to using nextTick? I'll try this. – majidarif Feb 13 '15 at 23:34
Here's an explanation of the difference: http://stackoverflow.com/a/15349865/1896761 – Austin Mullins Feb 13 '15 at 23:39
1

I see, but using `setImmediate` is causing some odd behavior. At times the messages are framed in delay. Thanks though but I found `nextTick` to be much better for my case. – majidarif Feb 13 '15 at 23:43

stack exceed on recursive function for framing tcp stream

1 Answers1