1

I have a large compressed .gz file (gigabyte level) that contains one large .txt file. Currently, I can add new lines to the end of the file without any problem. But I wonder if there's any way to update or modify lines as we iterate through the file? In my case, I have tab-separated lines where each tab-separated item is a column in my dataset. I need to add one more column to this data. One way is to create a new file, read the .gz file lines one by one, update them, and write them to the new file. However, this is not ideal since 1) the file is huge, and writing will take time, and 2) I may need to do this multiple times and more frequently.

Pedram
  • 2,421
  • 4
  • 31
  • 49
  • 2
    In general, this is not possible. But see [In-place modification of gzip files](https://stackoverflow.com/questions/69197360/in-place-modification-of-gzip-files) for some suggestions. – craigb Nov 04 '22 at 18:10
  • 1
    Nope. If you need to add a column to every line, the only way is to decompress, modify, and recompress the whole thing. – Mark Adler Nov 04 '22 at 23:12

0 Answers0