Node.js proxy, dealing with gzip compression

Question

I'm currently working on a proxy server where we in this case have to modify the data (by using regexp) that we push through it.

In most cases it works fine except for websites that use gzip as content-encoding (I think), I've come across a module called compress and tried to push the chunks that I receive through a decompress / gunzip stream but it isn't really turning out as I expected.

I was wondering if I am at all heading in the right direction, and if there are more modules out there to make my life easier (regarding gzip compression).

Greetz,

Benjamin

What do you mean "not turning out as expected"? With gzip (or any kind of lossless compression) it usually either works or doesn't work and you end up with useless gibberish. If that is the case, it is likely an encoding issue. You cannot send a `utf8` stream to a decompression module and expect it to decompress cleanly. The stream must remain in its native `Buffer` format. — MooGoo, Jan 03 '11 at 16:21

score 8 · Answer 1 · answered Mar 29 '11 at 01:00

8

If you think your proxy is just for filtering or modifying text, you feel verbose about gzip compression and decompression.

Another solution is simple.
Modify http request header. Then you can get plain text from server.

Remove 'Accept-Encoding' from http request header.

answered Mar 29 '11 at 01:00

SOnDDinG

81
1
2

4

req.headers['accept-encoding'] = '*;q=1,gzip=0'; // force uncompresed – user956584 Dec 27 '11 at 20:21

score 3 · Accepted Answer · edited May 23 '17 at 12:31

3

See here: Node.js proxy, dealing with gzip DEcompression

For an answer that covered most of my problems.

edited May 23 '17 at 12:31

Community

1
1

answered Jan 07 '11 at 11:59

M0rph3v5

945
1
11
23

score 0 · Answer 3 · edited May 23 '17 at 11:45

0

Have a look there: Node.js: Gzip compression?

There is a alternative to using node-compress, but this solution is also mentioned.

Cheers,

-stan

edited May 23 '17 at 11:45

Community

1
1

answered Jan 03 '11 at 16:21

spolu

680
5
11

Node.js proxy, dealing with gzip compression

3 Answers3