Write numbers to a file instead of string for data compression?

Question

I was encoding a simple text file using LZW algorithm in python. However, i realised that i can only write a string to a .txt file using the write() function, which itself occupies almost as much space. So is it possible to somehow just write actual integers to a file (maybe in a different format), to achieve proper compression ?

readfile = open("C:/Users/Dhruv/Desktop/read.txt", "r")
writefile = open("C:/Users/Dhruv/Desktop/write.txt", "w")
content = readfile.read()
length = len(content)

codes = []
for i in range(0, 256) :
    codes.append(str(chr(i)))

current_string = ""
for i in range(0, length) :
    temp = current_string + content[i]
    print(temp)
    if temp in codes :
        current_string += content[i]
    else :
        codes.append(current_string + content[i])
        writefile.write(str(codes.index(current_string)) + " ")
        current_string = str(content[i])
writefile.write(str(codes.index(current_string)) + " ")
readfile.close()
writefile.close();

Could you possibly mean *binary files*, opened for writing with mode `wb`... — Antti Haapala -- Слава Україні, Apr 03 '17 at 20:14
Agree with @AnttiHaapala, use "wb" and send it bytes() with a binary encoding. See http://stackoverflow.com/questions/20955543/python-writing-binary — RobertB, Apr 03 '17 at 20:41
I want to store integers greater than 255 also, how do i do that ? Also, i want to read them back as integers only — Dhruv Chadha, Apr 04 '17 at 07:19
You need an integer buffer to stuff in variable-sized codes, and you flush out bytes whenever more than 8 bits are present in the buffer. — SBS, Aug 05 '17 at 18:16

score 1 · Answer 1 · answered Apr 03 '17 at 20:15

If your data can be represented as Numpy array, the following function can write it as integer in a .txt file:

import numpy as np
def writer(_hd, _data):
    out_file_name = str(_hd) + '.csv'
    np.savetxt(out_file_name, _data, fmt='%i')
    return None

where _hd is the file name and _data is the numpy array. The fmt='%i' saves the data as integer; Other options are also available here.

Write numbers to a file instead of string for data compression?

1 Answers1