Compress in Java, decompress in Python?

Question

I am trying to compress a string in my Android app (Java) and then decompress it on the server with Python. I tried using the Inflater class, but then I get an "error: Error -3 while decompressing data: incorrect header check" on the python side. On the python side, I'm using zlib to decompress:

data_string = zlib.decompress(compressed_string)

On the Android side, we are using Deflater to compress:

String stringData = commentData == null ? sleepData.toString() : sleepData.toString() + commentData.toString();

byte[] bytes = stringData.getBytes("UTF-8");
int length = bytes.length;
Deflater deflater = new Deflater();
deflater.setInput(bytes);
deflater.finish();
byte[] compressedBytes = new byte[length];
deflater.deflate(compressedBytes);
deflater.end();

How can I decompress in python the string that the app is sending?

Thanks!

EDITS: This is what we are sending from Android and are feeding into the decompress command: (the dots represent just more of the same - there are a lot of zeros and then some "0b's" in the end: "78efbfbdefbfbdefbfbdefbfbd6e1b2....efbfbd190000000.....0000‌000000000b0b0b0b0b0b‌0b0b0b0b0b

Things we have tried:

-- data_string = zlib.decompress(decrypted,-zlib.MAX_WBITS), which gives a new error: "Error -3 while decompressing data: invalid stored block lengths"

-- data_string = zlib.decompressobj(decrypted,15+32) and data_string = zlib.decompress(compressed_string, 16+zlib.MAX_WBITS) which both give the initial error "invalid header check".

How are you sending the data from the Android/Java side to the Server/Python side? — Robᵩ, Oct 25 '16 at 18:27
Related: http://stackoverflow.com/questions/2424945/are-zlib-compress-on-python-and-deflater-deflate-on-java-android-compatible — Robᵩ, Oct 25 '16 at 18:33
That should work, since `Deflater` by default produces the zlib format, and `zlib.decompress()` by default is expecting the zlib format. Can you provide the first several bytes of what you are feeding to `zlib.decompress()`, in hex? — Mark Adler, Oct 25 '16 at 19:12
This is what the string looks like: (the dots represent just more of the same - there are a lot of zeros and then some "0b's" in the end: "78efbfbdefbfbdefbfbdefbfbd6e1b2....efbfbd190000000.....0000000000000b0b0b0b0b0b0b0b0b0b0b" — Maria, Oct 27 '16 at 16:49
as for the sending of the data - we first compress, then encrypt with AES256 on the Android side, and then send it to the server, where we decrypt, then try to decompress. Before we added the compression, the encrypt/decrypt part was working fine. — Maria, Oct 27 '16 at 16:55

score 1 · Accepted Answer · edited May 23 '17 at 10:32

1

Add 16+zlib.MAX_WBITS or 15 + 32 as the second [optional] wbits arg in decompress: https://docs.python.org/2/library/zlib.html#zlib.decompress

data_string = zlib.decompress(compressed_string, 16+zlib.MAX_WBITS)

EDIT: Depending on how you're passing the compressed bytes to Python, it's likely your issue is with how Python is reading your compressed bytes.

I recommend using Base64 encoding on the compressed bytes in Java, then decoding the base64 in Python then decompressing those bytes in this vein...

Java:

String inputString = "blahblahblah";
byte[] input = inputString.getBytes("UTF-8");

// Compress the bytes
byte[] output = new byte[100];
Deflater compresser = new Deflater();
compresser.setInput(input);
compresser.finish();
int compressedDataLength = compresser.deflate(output);
compresser.end();
System.out.println(Base64.getEncoder().encodeToString(output));

Python:

import base64
import zlib
base64_binary = b"eJxLyknMSIJiAB8CBMYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="
deflate_bytes=base64.decodebytes(base64_binary)
zlib.decompress(deflate_bytes)
# b'blahblahblah'

Python decompressing gzip chunk-by-chunk

Byte Array in Python

edited May 23 '17 at 10:32

Community

1
1

answered Oct 25 '16 at 18:40

Garren S

5,552
3
30
45

Thanks for the answers! Adding 16+zlib.MAX_WBITS or 15 + 32 as the option parameters didn't work. It's giving the same error. – Maria Oct 27 '16 at 16:46
By the way, we also tried "data_string = zlib.decompress(decrypted, -zlib.MAX_WBITS)", and that gave us error "Error -3 while decompressing data: invalid stored block lengths" – Maria Oct 27 '16 at 17:02
@Maria http://stackoverflow.com/questions/3122145/zlib-error-error-3-while-decompressing-incorrect-header-check Do any of those examples work? I suspect not given your invalid stored block lengths error above. Could you provide samples of what you're trying to do exactly? – Garren S Oct 27 '16 at 18:19
no, they didn't work either! I am not sure how to give more examples, that's literally all the code we have related to decompressing. – Maria Oct 27 '16 at 22:09
@Maria I updated my answer to recommend Base64 encoding to ensure the data being read in Python is the same as the data Java is outputting. Note my example assumes Python 3. – Garren S Oct 27 '16 at 22:38
1

thank you!we got it working once we change the java code to be like what you suggested, but java for android didn't have the getEncoder() method so we used: int length = bytes.length; Deflater deflater = new Deflater(); deflater.setInput(bytes); deflater.finish(); byte[] compressedBytes = new byte[length]; deflater.deflate(compressedBytes); deflater.end(); return Base64.encodeToString(compressedBytes, Base64.DEFAULT); And on the python side, I don't have python 3 yet, so we used: deflate_bytes=base64.decodestring(base64_binary) – Maria Nov 03 '16 at 16:57

Compress in Java, decompress in Python?

1 Answers1