33

JSON.stringify is obviously not very space efficient. For example, [123456789,123456789] occupy 20+ bytes when it could need just around 5. Does websocket compress its JSONs before sending to stream?

Joakim Erdfelt
  • 46,896
  • 7
  • 86
  • 136
MaiaVictor
  • 51,090
  • 44
  • 144
  • 286

4 Answers4

55

WebSocket is, at its heart, just a set of framing for TEXT or BINARY data.

It performs no compression on its own.

However, the WebSocket spec allows for Extensions, and there have been a variety of compression extensions in the wild (the formalized specs for one of these is finalized).

As of today (August 2018) the accepted compression spec is permessage-deflate.

Some of the extensions seen in the wild:

  • permessage-deflate - the name of the formalized spec for using deflate to compress entire messages, regardless of the number of websocket frames.
  • x-webkit-deflate-frame - an early proposed compression that compresses each raw websocket data frame. Seen in use by Chrome and Safari. (now deprecated in Chrome and Safari)
  • perframe-deflate - a renamed version of the above compression. Seen in use by various websocket server implementations, and also briefly showed up in various WebKit based clients. (Completely deprecated in modern browsers, but does still show up in various WebSocket client libraries)

Of note, the permessage-deflate extension is the first in a line of PMCE (Per-Message Compression Extensions) that will eventually include other compression schemes (ones being discussed are permessage-bzip2, permessage-lz4, and permessage-snappy)

Community
  • 1
  • 1
Joakim Erdfelt
  • 46,896
  • 7
  • 86
  • 136
  • As Joakim mentions, `permessage-deflate` is not yet formally issued. FWIW, Chromium has it (but you need to manually enable) and they "intent to ship" soon: https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/-B4uhxryIG4 – oberstet Oct 10 '13 at 16:05
  • Thanks @Joakim Erdfelt, but does permessage-deflate use gzip? I am trying to compress gzip data over the websocket pipe. – NiCk Newman Aug 23 '15 at 03:58
  • 1
    @NiCkNewman the extension `permessage-deflate` is the `deflate` algorithm, gzip is different. If you are doing this yourself, outside of an extension, then note that you can only use BINARY messages (for the websocket protocol), as compressed TEXT messages would violate the rfc-6455 spec (requiring that TEXT messages be UTF-8) – Joakim Erdfelt Aug 24 '15 at 22:43
  • 9
    Can you update your answer as per updates of today (I mean end of 2016) – Ravinder Payal Aug 03 '16 at 16:08
23

Does websocket compress its JSONs before sending to stream?

The short answer is: sometimes, but you can't depend on it.

As stated aptly by Joakim Erdfelt, Websocket connections support both Text and Binary messages.

JSON is just one way to transport data, with the advantage of versatility and ease of use (while being wasteful as far as space is concerned).

You can easily transfer binary data using the Websocket API, eliminating bandwidth overhead at the expense of other concerns (such as endieness, word length, parsing, etc').

Many browsers also support Websocket message compression as an extension to the Websocket protocol (although the server might not support the extension).

Extensions are negotiated using the Sec-WebSocket-Extensions HTTP header. The negotiations are usually implemented by the client / server without offering a public API to control them.

Right up until 2015, there were a number of approaches and implementations in the wild, but since December 2015 RFC 7692 is the only real contender for message compression and things are much clearer.

RFC 7692 compresses the whole message before wrapping (and perhaps fragmenting) it in Websocket "packets", making it easier to implement than some previous compression schemes.

The current draft offers a permessage-foo compression negotiation scheme (where foo is the requested / supported compression).

I only experienced the permessage-deflate extension myself.

Do notice that extension negotiations are optional, meaning that potential network clients are usually allowed to negotiate connections without compression, even if your server supports the extension.

Moreover, RFC 7692 supports selective compression, meaning some messages might be compressed while others aren't compressed...

... for example, [123456789,123456789] might be sent as is, since it's length indicates it isn't likely to be worth the compression effort.

Support for permessage-deflate (RFC 7692), August 8th, 2017:

This is a combination of the information in the comments, last updated August 8th, 2017.

If I missed anything, add them here and update the date.

Known Browser Support

  • Chrome since 32
  • Mozilla Firefox since 37
  • Safari - unsupported (10.1.2 seems to use x-webkit-deflate-frame)
  • Microsoft Edge - unsupported as of version 40

Known Server Support

John Zwinck
  • 239,568
  • 38
  • 324
  • 436
Myst
  • 18,516
  • 2
  • 45
  • 67
  • It looks like permessage-deflate is supported in since Google Chrome 32 and Mozilla Firefox 37. I see no evidence that Microsoft Edge supports it as of August 2017 (Edge 40). On the Python server side, it seems only Crossbar.io (Autobahn), wspy and Tornado support it as of August 2017. – John Zwinck Aug 08 '17 at 08:08
  • @JohnZwinck - to add to the current state of affairs, Safari uses `x-webkit-deflate-frame` and on the server side, Faye supports `permessage-defalte` on node.js. – Myst Aug 08 '17 at 08:14
  • 1
    @JohnZwinck I added the information to the answer in a way that should make it easy to update by future editors. – Myst Aug 08 '17 at 08:26
  • 1
    Commenters: don't be nervous about updating the info in the original post. That's part of what StackOverflow is for :) – acjay Apr 06 '18 at 13:08
  • Doesn't seem like Safari works as of Jan 2020 (where Firefox does). Safari: "WebSocket connection to 'ws://192.1.1.100:8080/ws' failed: Failed to compress frame" – cloudsurfin Jan 28 '20 at 05:42
1

You could use Unishox compression technique for compressing text sent over Websockets. However this is not integrated with the spec and the implementor is responsible for compressing one one side and decompressing on the other side.

Unishox achieves compression by assigning fixed prefix-free codes for each letter in the given Character Set based on known frequency of popular symbols (entropy coding). It also encodes repeating letter sets separately (dictionary coding). For Unicode characters, delta coding is used. More information is available in this article.

So far it has been implemented in C and Javascript.

Disclaimer: I am the developer of Unishox.

Arundale Ramanathan
  • 1,781
  • 1
  • 18
  • 25
  • 1
    I considered using it, however I changed my mind after testing the speed ... I wish the paper had given benchmark numbers for this instead of saying "it's slow but shouldn't matter in most cases". I think it could be improved by giving the option to disable some of the slow features, like the sequence repetition. – Octo Poulos Jun 03 '22 at 15:30
  • @OctoPoulos Hi thanks for the feedback! Not sure what you were planning to use it for. It works best for strings of length less than 1kb and in such cases performance should not matter much (IMO). But yes, I am planning to do away with some of the features that are rarely used in the next version. – Arundale Ramanathan Jun 05 '22 at 12:48
  • "performance should not matter" really? Compression is about minimizing the data payload at the expense of processing power. You might be using some cool techniques to shave off a few bytes, but if it kills my CPU for 2 seconds to save 50 bytes then the algorithm is useless in the real world. Current compression algorithms such as zlib are state of the art and there is little point in compressing something that is already so short anyway. – Phil Aug 18 '22 at 22:09
  • I am working on improving performance, but even now it isn't as bad as 2 seconds to save 50 bytes – Arundale Ramanathan Aug 19 '22 at 02:55
-5

Websockets send raw bytes.
They don't know or care what those bytes represent.

If you want to compress data, you'll need to compress it yourself before sending it down the wire.

Note that Chrome supports gzip for websocket connections. (assuming your server also does)

SLaks
  • 868,454
  • 176
  • 1,908
  • 1,964
  • Does gzip compress numeric strings like those (I guess no)? – MaiaVictor Oct 10 '13 at 14:35
  • 1
    There is no "gzip" WebSocket compression in Chrome or elsewhere (other browser). – oberstet Oct 10 '13 at 16:08
  • 1
    @oberstet: Are you sure? http://stackoverflow.com/questions/11646680/could-websocket-support-gzip-compression#comment15442271_11647130 – SLaks Oct 10 '13 at 16:22
  • 5
    gzip is a _file format_ that uses deflate as it's compression algorithm. deflate is a combination of LZ77 and Huffman coding. The (old) per-frame compression WS extension and the new per-message WS compression extension use deflate. – oberstet Oct 10 '13 at 21:48