0

Background: I am testing compression ratio for our device. From device the data is sent as Json payloads. It's a JArray format. I will get the size of this data in Bytes. It goes to converter. In converter this data is been compressed using Zlib library implemented in NodeJS. I don't have access to this code. And I don't know NodeJS. I know only C#.

Is that okay to do compression using SharpZipLib or any other GZip compression library in .Net to replicate the job of the converter. How much accurate it will be? Will the compression vary largely because of using different libraries?

KBNanda
  • 595
  • 1
  • 8
  • 25
  • 1
    it is *not* the case that there is one and only one way of applying gzip/deflate/etc - and yes: different choices can give different lengths and thus different compression ratios; do not rely on always getting the exact same compressed bytes - the only relevant question is whether they *decompress* back to the original values; the ratios will usually be *very similar* between implementations – Marc Gravell May 25 '21 at 09:50
  • 1
    Compression can vary with same tool depending on the sector size of the drive you are using. Many algorithms use a fast approach and compress one sector of the disk at a time. I've had issues where I was saving a ZIP output on a UNIX File Server and then wasn't able to unzip using a window unzip tool. The tools that was making zip was part of a MAKE utility. We had to change our build instructions to make zip on windows machine and then copy the results to the Unix File Server. – jdweng May 25 '21 at 10:03
  • @MarcGravell I was asked to test the compression ratio implemented using zlib library. I don't have the access to the compression logic. Hence to get the right result I need to use the same library (means zlib on NodeJS here) and same mechanism of compression by knowing that – KBNanda May 25 '21 at 10:16

2 Answers2

1

The compression ratio will be mostly depending on the compression algorithm and what compression settings are used. Deflate is one of the most common. I would not expect any significant difference in size.

But even if the same algorithm is used there might be different headers used by different libraries. If you want backward compatibility you need to have a library that is compatible. See also zip vs gzip vs zlib.

The best way to check if there is any size difference or backward compatibility issues is to test it.

JonasH
  • 28,608
  • 2
  • 10
  • 23
1

If the libraries are well implemented, you should not see a huge difference in compression ratios at the default settings. They will all have some sort of compression level setting that you can vary, which adjusts how much CPU time they will invest in trying to compress better. So even for a single one of those compressors, you will in fact see some noticeable difference at different compression levels.

There have been bad implementations. Notably .NET before zlib was integrated into it. It had terrible compression ratios due to a clueless implementation.

What really matters is that compression followed by decompression is lossless. If you make sure your use of the libraries obeys that, then you're fine.

Mark Adler
  • 101,978
  • 13
  • 118
  • 158
  • Thanks. In one of the components in our project they have implemented compression using Zlib library in NodeJs. If I use Zlib for .net will I achieve the same compression ratio or it differs? – KBNanda May 25 '21 at 18:40
  • 1
    I might guess the same, but I don't know. How about you try it? – Mark Adler May 25 '21 at 19:45