2

I need to generate the tar.gz (or tar.bz2) archive in Python, where contents of each file is generated "on the fly". The ideal solution should look as follows:

import hypothetic_extended_tar_lib as tl
tf1=tl.open("myarchive.tar.bz2",mode="wb")
df1=tf1.newfile("file1.bin")
#Now get data from the DAQ1
for rec_num in xrange(1,1000):
   data=daq1.get_data(rec_num)
   df1.write(data)
df1.close()
df2=tf1.newfile("file2.bin")
#Get some data from the second DAq
for rec_num in xrange(1,500):
   data=daq2.get_data(rec_num)
   df2.write(data)
df2.close()
tf1.close()

I now, that it can be done by creating files on disk and adding them later on. But due to insufficient storage space, I have to compress data on the fly, and additionally I want to have them placed in a single compressed tar archive.

wzab
  • 788
  • 7
  • 24
  • Not really, he's looking to generate the files and compress them while still in memory. He doesn't want any uncompressed files written to disk. – Brian Jun 28 '13 at 18:14
  • @jpe I think you found the question it is actually a duplicate of! I was thinking something along the lines of what has been said in that post but didn't have the time while at work to look into it. – Brian Jun 28 '13 at 18:36
  • 1
    Possible duplicate of http://stackoverflow.com/questions/1389681/how-to-write-a-large-amount-of-data-in-a-tarfile-in-python-without-using-tempora – jpe Jun 28 '13 at 18:48

0 Answers0