WebSocket Data Parsing with Node.js

Question

Websocket on Client:

socket.send('helloworld');

Websocket on Node.js:

socket.ondata = function(d, start, end){
     // I suppose that the start and end indicates the location of the 
     // actual data 'hello world' after the headers
     var data = d.toString('utf8', start, end);
     
     // Then I'm just printing it
     console.log(data);
});

but I'm getting this: �����]���1���/�� on the terminal :O

I have tried to understand this doc: https://www.rfc-editor.org/rfc/rfc6455#section-5.2 but it's hard to understand because I don't know what should I work with, I mean I can't see the data even with toString?

I have tried to follow and test with this questions answer How can I send and receive WebSocket messages on the server side? but I can't get it to work, with this answer I was getting an array like this [false, true, false, false, true, true, false] etc... and I don't really know what to do with it.. :\

So I'm a bit confused, what the hell should I do after I get the data from the client side to get the real message?

I'm using the original client side and node.js API without any library.

Answer 1

Which node.js library are you using? Judging by the fact that you are hooking socket.ondata that looks like the HTTP server API. WebSockets is not HTTP. It has an HTTP compatible handshake so that the WebSocket and HTTP service can live on the same port, but that's where the similarity ends. After the handshake, WebSockets is a framed full-duplex, long-lived message transport more similar to regular TCP sockets than to HTTP.

If you want to implement your own WebSocket server in node.js you are going to want to use the socket library directly (or build on/borrow existing WebSocket server code).

Here is a Node.js based WebSocket server that bridges from WebSocket to TCP sockets: https://github.com/kanaka/websockify/blob/master/other/websockify.js Note that it is for the previous Hixie version of the protocol (I haven't had opportunity or motivation to update it yet). The modern HyBI version of the protocol is very different but you might be able to glean some useful information from that implementation.

Answer 2

You can in fact start with Node's HTTP API. That is exactly what I did when writing the WebSocket-Node module https://github.com/Worlize/WebSocket-Node

If you don't want to use an existing WebSocket Library (though you really should just use an existing library) then you need to be able to parse the binary data format defined by the RFC. It's very clear about the format and exactly how to interpret the data. From each frame you have to read in all the flags, interpret the frame size, possibly read the masking key, and unmask the contents as you read them from the wire.

That is one reason you're not seeing anything recognizable... in WebSockets, all client-to-server communications is obfuscated by applying a random mask to the contents using XOR as a security precaution against possibly poisoning the cache of older proxy servers that don't know about websockets.

Answer 3

Here is a solution from this post:

https://medium.com/hackernoon/implementing-a-websocket-server-with-node-js-d9b78ec5ffa8

parseMessage(buffer) {
    const firstByte = buffer.readUInt8(0);
    //const isFinalFrame = Boolean((firstByte >>> 7) & 0x1); 
    //const [reserved1, reserved2, reserved3] = [ Boolean((firstByte >>> 6) & 0x1), 
    Boolean((firstByte >>> 5) & 0x1), Boolean((firstByte >>> 4) & 0x1) ]; 
    const opCode = firstByte & 0xF; 
    // We can return null to signify that this is a connection termination frame 
    if (opCode === 0x8) 
        return null; 
    // We only care about text frames from this point onward 
    if (opCode !== 0x1) 
        return; 
    const secondByte = buffer.readUInt8(1); 
    const isMasked = Boolean((secondByte >>> 7) & 0x1); 
    // Keep track of our current position as we advance through the buffer 
    let currentOffset = 2; let payloadLength = secondByte & 0x7F; 
    if (payloadLength > 125) { 
        if (payloadLength === 126) { 
            payloadLength = buffer.readUInt16BE(currentOffset); 
            currentOffset += 2; 
        } else { 
            // 127 
            // If this has a value, the frame size is ridiculously huge! 
            //const leftPart = buffer.readUInt32BE(currentOffset); 
            //const rightPart = buffer.readUInt32BE(currentOffset += 4); 
            // Honestly, if the frame length requires 64 bits, you're probably doing it wrong. 
            // In Node.js you'll require the BigInt type, or a special library to handle this. 
            throw new Error('Large payloads not currently implemented'); 
        } 
    }

    const data = Buffer.alloc(payloadLength);
    // Only unmask the data if the masking bit was set to 1
    if (isMasked) {
        let maskingKey = buffer.readUInt32BE(currentOffset);
        currentOffset += 4;

        // Loop through the source buffer one byte at a time, keeping track of which
        // byte in the masking key to use in the next XOR calculation
        for (let i = 0, j = 0; i < payloadLength; ++i, j = i % 4) {
            // Extract the correct byte mask from the masking key
            const shift = j == 3 ? 0 : (3 - j) << 3; 
            
            const mask = (shift == 0 ? maskingKey : (maskingKey >>> shift)) & 0xFF;
            // Read a byte from the source buffer 
            const source = buffer.readUInt8(currentOffset++); 
            // XOR the source byte and write the result to the data 
            data.writeUInt8(mask ^ source, i); 
        }
    } else {
        // Not masked - we can just read the data as-is
        buffer.copy(data, 0, currentOffset++);
    }

    return data 
}

Answer 4

There are two things -

1. Which node.js version are you using? I have never seen a data event with start and endpt. The emitted event is just data with buffer/string as an argument.

2. More importantly, if you are hooking on to the HTTP socket you should take care of the HTTP Packet. It contains hearders, body and a trailer. There might be garbage in there.