HTTP - how to send multiple pre-cached gzipped chunks?

Question

Lets say I have 2 individually gziped html chunks in memory. Can I send chunk1+chunk2 to HTTP client? Does any browser supports this? Or there is no way to do this and I have to gzip the whole stream not individual chunks?

I want to serve to clients for example chunk1+chunk2 and chunk2+chunk1 etc (different order) but I don't want to compress the whole page every time and I dont want to cache the whole page. I want to use precompressed cached chunks and send them.

nodejs code (node v0.10.7):

// creating pre cached data buffers
var zlib = require('zlib');
var chunk1, chunk2;
zlib.gzip(new Buffer('test1'), function(err, data){
  chunk1 = data;
});
zlib.gzip(new Buffer('test2'), function(err, data){
  chunk2 = data;
});


var http = require('http');
http.createServer(function (req, res) {
      res.writeHead(200, {'Content-Type': 'text/plain', 'Content-Encoding': 'gzip'});
      // writing two pre gziped buffers
      res.write(chunk1); // if I send only this one everything is OK
      res.write(chunk2); // if I send two chunks Chrome trying to download file
      res.end();
}).listen(8080);

When my example server returns this kind of response Chrome browser display download window (it doesnt understand it :/

Read few times the question - it's unclear. What are you talking about? GZip? What do you mean? About the underlying HTTP compression? — Poni, May 24 '13 at 16:59
No, you can't concatenate gzipped blocks. Compression does not work this way. gzip is quite a fast algorithm, could it be that you are trying to over-engineer/over-optimize this? — Tomalak, May 24 '13 at 17:05
Your finding suggests that, while legal, browsers don't support this. If you run this through Fiddler and try to use the Transformer to decompress, what do you see? — EricLaw, May 24 '13 at 21:58
HTTP/1.1 200 OK Content-Type: text/plain Content-Encoding: gzip Date: Fri, 24 May 2013 22:18:16 GMT Connection: keep-alive Transfer-Encoding: chunked 19 ��+I-.1��ܲ�� 7 +I-.1� 0 After decoding: HTTP/1.1 200 OK Content-Type: text/plain Date: Fri, 24 May 2013 22:18:16 GMT Connection: keep-alive Content-Length: 5 test1 — Tereska, May 24 '13 at 22:19

score 5 · Answer 1 · edited Oct 07 '21 at 05:54

5

I haven't tried it, but if the http clients are compliant with RFC 1952, then they should accept concatenated gzip streams, and decompress them with the same result as if the data were all compressed into one stream. The HTTP 1.1 standard in RFC 2616 does in fact refer to RFC 1952.

If by "chunks" you are referring to chunked transfer encoding, then that is independent of the compression. If the clients do accept concatenated streams, then there is no reason for chunked transfer encoded boundaries to have to align with the gzip streams within.

As to how to do it, simply gzip your pieces and directly concatenate them. No other formatting or preparation is required.

edited Oct 07 '21 at 05:54

Community

1
1

answered May 24 '13 at 17:26

Mark Adler

101,978
13
118
158

So, if concatenation is not supported by the browsers, are there any options to combine gzipped pieces together, without gunzipping and regzipping them? Something like playing with zlib headers or else? – Nikolai Gorchilov Aug 29 '13 at 17:38
Yes, that is possible. You don't have to re-gzip, but you do have to ungzip the first stream in order to disect the end of the stream and patch in the next one. See [gzappend.c](https://github.com/madler/zlib/blob/master/examples/gzappend.c). – Mark Adler Aug 29 '13 at 19:28
3

For anyone who's curious, browsers do not seem to support concatenated gzip steams. I tried in Firefox without success and this Chrome bug indicates no Chrome support: http://code.google.com/p/chromium/issues/detail?id=20884 – Evan Shaw Sep 17 '13 at 22:16
1

Interesting. From the link, it appears that only Opera is compliant with the standard. – Mark Adler Sep 17 '13 at 22:18

HTTP - how to send multiple pre-cached gzipped chunks?

1 Answers1

Linked