Stack Overflow
Stack Overflow

Questions tagged [lz4]

LZ4 is a lossless data compression algorithm that is focused on compression and decompression speed

LZ4 is a lossless data compression algorithm that is focused on compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes.

Use this tag for questions that have issues with the implementations in one of the native languages or binding like c, c++, java, python.

LZ4 Repository

LZ4 Homepage

Adapted From Wikipedia

166 questions
53
votes
1 answer

Difference: LZ77 vs. LZ4 vs. LZ4HC (compression algorithms)?

I understand the LZ77 and LZ78 algorithms. I read about LZ4 here and here and found code for it. Those links described the LZ4 block format. But it would be great if someone could explain (or direct me to some resource explaining): How LZ4 is…
ghost204nit
  • 682
  • 1
  • 7
  • 13
22
votes
4 answers

Comparison between lz4 vs lz4_hc vs blosc vs snappy vs fastlz

I have a large file of size 500 mb to compress in a minute with the best possible compression ratio. I have found out these algorithms to be suitable for my use. lz4 lz4_hc snappy quicklz blosc Can someone give a comparison of speed and…
Sayantan Ghosh
  • 998
  • 2
  • 9
  • 29
19
votes
4 answers

How to use tar with lz4?

How to use tar and filter the archive through LZ4? Or any available tools? It looks cumbersome to use tar cvf folderABC.tar folderABC && lz4c -c0 folderABC.tar. PS: *nix environment
Daniel
  • 2,576
  • 7
  • 37
  • 51
11
votes
1 answer

How do I decide between LZ4 and Snappy compression?

I need to select a compression algorithm when configuring "well-known application". Also, as part of my day job, my company is developing distributed application that deal with a fair amount of data. We've been looking into compressing data to try…
user5994461
  • 5,301
  • 1
  • 36
  • 57
11
votes
2 answers

LZ4 library decompressed data upper bound size estimation

I'm using LZ4 library and when decompressing data with int LZ4_decompress_safe (const char* source, char* dest, int compressedSize, int maxDecompressedSize); I want to estimate maximum decompressed data size. But I can not find reverse function…
bobeff
  • 3,543
  • 3
  • 34
  • 62
11
votes
1 answer

Using lz4 compression in javascript without Node.js

I am trying to get LZ4 decompression to work client-side in pure JavaScript. I found this JavaScript library but it's meant to be used with Node.js. Within the same repository I found a library which can in theory be used in the browser. The problem…
Tarelli
  • 634
  • 7
  • 18
9
votes
0 answers

How to point to use lz4 for compression

Ok, so I need to build TWRP image but I need to point to it to use lz4 instead of gzip format. How to do that I guess it should be in boardconfing.mk under BOARD_MKBOOTIMG_ARGS?
Једноруки Крстивоје
  • 183
  • 5
9
votes
3 answers

OverflowError with Pandas to_hdf

Python newbie here. I am trying to save a large data frame into HDF file with lz4 compression using to_hdf. I use Windows 10, Python 3, Pandas 20.2 I get the error “OverflowError: Python int too large to convert to C long”. None of the machine…
Ron
  • 91
  • 1
  • 3
9
votes
1 answer

LZ4 compression algorithm explanation

Description from Wikipedia: The LZ4 algorithm represents the data as a series of sequences. Each sequence begins with a one byte token that is broken into two 4 bit fields. The first field represents the number of literal bytes that are to be…
ade
  • 379
  • 1
  • 6
  • 15
8
votes
3 answers

Failed to load Python extension for LZ4 support. LZ4 compression will not be available

I am new to ROS, I just opened a terminal and typed roscore and another terminal and typed rostopic node and I got this error that says Failed to load Python extension for LZ4 support. LZ4 compression will not be available. I searched and went to…
Abdelrahman Emam
  • 385
  • 1
  • 6
  • 15
7
votes
1 answer

How to decompress lz4 file

I have a folder/file which is with the extension abc.tar.lz4 I want to be able to view the contents of this file but I have been unable to do so. I am using MACOS and I installed lz4 via my terminal using the command: brew install lz4 How can I…
Dev.D
  • 407
  • 2
  • 4
  • 12
7
votes
1 answer

Execute linux command on Centos using dotnet core

I'm running a .NET Core Console Application on CentOS box. The below code is executing for normal command like uptime, but not executing for lz4 -dc --no-sparse vnp.tar.lz4 | tar xf - Logs.pdf: try { var connectionInfo = new…
Itniv
  • 112
  • 2
  • 2
  • 11
7
votes
4 answers

How to compress and decompress a file using lz4?

I want to compress and decompress a file using lz4 algorithm in Go. Is there any package available to do this? I searched and found a package called https://github.com/pierrec/lz4 I am new Go and I cannot figure out how to use this package to…
Dharani Dharan
  • 624
  • 1
  • 7
  • 18
6
votes
1 answer

Compress a folder into a single file using lz4 command line

I need to compress the all of the contents of a directory into a single lz4 archive, i couldn't figure out a way to do that. I already have read all the available parameters, still no success. Kindly suggest me how can i achieve this.
Pravesh Agrawal
  • 869
  • 1
  • 11
  • 27
6
votes
3 answers

Java LZ4 compression using Input/Output streams

I'm using jpountz LZ4 to try and compress files and I want to read in and output files using Java file input and output streams. I've tried to find a solution online but theres nothing, I found a previous stackoverflow question on how to implement…
user3758298
  • 380
  • 5
  • 17
1
2 3
…
10 11