Websocket frame size limitation

Question

I'm sending huge chunks of JSON data through websockets. The JSON may have over 1000 entries. Due to the frame size limitation, the Websocket protocol automatically splits the JSON into frames, which cannot be helped. As we cannot change the frame size of websockets.

The problem:

When I try to evaluate my JSON using JSON.parse it gives me a parsing error which is obvious because the frames are not complete JSON objects. All this happens in the Websocket onmessage event callback. How can I recieve the huge JSON in differnt frames and still be able to parse it? I have tried to concat the frames in onmessage still the error persists.

Side question:

How to concatinate a broken JSON properly?

Add the buffer size params from this list into your web.xml https://github.com/Atmosphere/atmosphere/blob/master/modules/cpr/src/main/java/org/atmosphere/cpr/ApplicationConfig.java — Gaurav Bhor, Oct 20 '14 at 09:30

score 45 · Answer 1 · edited Oct 07 '21 at 05:46

45

A single WebSocket frame, per RFC-6455 base framing, has a maximum size limit of 2^63 bytes (9,223,372,036,854,775,807 bytes ~= 9.22 exabytes) (correction by @Sebastian)

However, a WebSocket message, made up of 1 or more frames, has no limit imposed on it from the protocol level.

Each WebSocket implementation will handle message and frame limits differently. Such as setting maximum messages sizes for whole message (usually for memory consumption reasons), or offering streaming options for large messages to better utilize memory.

But in your case, it is likely that your chosen WebSocket implementation has a bug and is improperly splitting up the JSON message into multiple messages, instead of multiple frames. You can use the network inspection tooling in Chrome or an external tool like Wireshark to confirm this behavior.

edited Oct 07 '21 at 05:46

Community

1
1

answered Dec 18 '13 at 12:42

Joakim Erdfelt

46,896
7
86
136

2

It is not splitting the JSON in multiple messages, it is splitting into multiple frames. I know that only because I have inspected using the Chrome tool. How do I handle it? I'm using Atmosphere Framework. – Gaurav Bhor Dec 18 '13 at 14:27
Have you try to change the websocket's buffer size? Let's have the discussion on the Atmosphere mailing list, but my suspicion is you need to increase the server websocket buffer size. – jfarcand Dec 18 '13 at 15:10
12

9.22 exabytes... guess I'm gonna need a bigger frame buffer. – bvj Feb 06 '20 at 07:20

score 7 · Answer 2 · edited Dec 14 '17 at 08:55

7

var wsServer = new websocket.server({
            httpServer: server,
            maxReceivedFrameSize: 131072,
            maxReceivedMessageSize: 10 * 1024 * 1024,
            autoAcceptConnections: false
        });

Change the default maxFrameSize & MessageSize

edited Dec 14 '17 at 08:55

sheldonzy

5,505
9
48
86

answered Dec 14 '17 at 08:29

Dc_Neo

101
1
4

i get this same error. but im running a client. is this applicable to the client as well..not finding much docs on it – mike01010 Oct 16 '22 at 22:40

score 3 · Answer 3 · answered Mar 25 '16 at 18:10

Since you are dealing with WS, which is low-level, you need to create an application protocol that deals with data that is sent over multiple WS frames. It is up to you to concatenate the data that is in each WS frame (btw, don't concat the frames... concat the data in each frame).

Basically you are reinventing a file transfer protocol.

Websocket frame size limitation

3 Answers3

Linked