Detect complete data received on 'data' Event in Net module of Node.js

Question

Currently when we send a large data over TCP data event will receive it in chunks. Is there a way where we able to detect that all data is received and return complete data.

Answer 1

Currently when we send a large data over TCP data event will receive it in chunks. Is there a way where we able to detect that all data is received and return complete data.

TCP is just a continuous stream of data. There's no start or end to a given transmission at the TCP level unless you close the connection and use a closed connection as a signal that the data finished.

So, if you want to know where a given chunk of data starts and stops, you need to design a way within the data you send in order to know that (essentially build your own little mini wire format or protocol). There are a zillion different ways to do that and that's one of the reason we have so many different protocols built on top of TCP. The two simplest schemes are to:

1. Send a length of your packet and then send that many bytes. The recipient then reads the length and knows when it then reads that many bytes, it has the whole chunk.

2. Use some sort of delimiter that won't appear in the actual data. For example, some simple protocols use a linefeed as a delimiter. You send a bunch of text and then terminate it with a linefeed. The receipient reads data until they get a linefeed and that tells them they have a complete chunk of the data. There are many possible delimiters depending upon what the type of data is.

Other protocols such as webSocket or socket.io have message-based paradigms built into them doing this work for you. You send a message at one end and then receive a whole message at the other end.

Some options are more or less appropriate based on the type of data (text/binary) you're sending and the length of the data and what the character of the data is (whether there are possible delimiters that won't be in the actual data).