1

I would like to create a tar archive in Java. I have files which are constantly being created and I'd like a worker thread to take a reference to those files from a queue and copy them into the archive.

I tried using Apache Compression library's TarArchiveOutputStream to do this, but I do not wish to keep the archive open for the entire duration of the program (since unless i finalize the archive, it can become corrupted - so i'd rather append to it in batches), and I haven't found a good way to append to an existing tar archive with their library (They do have the "ChangeSetPerformer" class, but it basically just creates a new tar and needs to copy the old one entirely, which isn't good for me, performance wise).

I also need the library to not have a low limit for the size of the archive (i.e. 4g or so is not enough), and i'd rather avoid having to actually compress the archive.

Any suggestions would be greatly appreciated! thank you.

user976850
  • 1,086
  • 3
  • 13
  • 25

2 Answers2

2

You run here in a limitation of tar: http://en.wikipedia.org/wiki/Tar_(file_format)#Random_access

because of that it is hard to add or remove single files without copying the whole archive.

Mnementh
  • 50,487
  • 48
  • 148
  • 202
  • My understanding from that section of the article is that it is actually easy to add a single file to a tar archive without copying it, as you can just stick it on the end without updating a TOC. – Samuel Edwin Ward Mar 19 '12 at 13:02
  • Why would I need random access though, if all I want to do is append (not caring about the order of the files within)? Also, what other format would you recommend that I use instead? – user976850 Mar 19 '12 at 13:06
0

I use a library called Java Tar: http://www.trustice.com/java/tar/

It's worked for me. In that package, look for:

http://www.gjt.org/javadoc/com/ice/tar/TarArchive.html#writeEntry(com.ice.tar.TarEntry, boolean)

Which lets you add an entry to the file without using a stream at the user level. I don't know about file size - but it would be a simple matter to test this aspect.

Mick Sear
  • 1,549
  • 15
  • 25