Python - archiving file before moving to another directory?

Question

Let say I have this code that moves files in another directory:

import os, shutil

    try:
        os.chdir(rec.gec_daily_dir)
        direct = os.listdir(/src/dir/)
        for f in direct:
            if f[-3:] == 'xls':
                shutil.move(f, /archive/dir/)
    except TypeError:
        print "Directory or file not found"

So how could I archive (or compress) file just before moving it to antoher directory (does not matter zip or tar.gz)? (original file should only be in archive state, not like when you manually archive it, it leaves original file un-archived)

I saw there a lib tarfile, where you could do something like that, but I saw some examples ofr archiving whole directory, but what I need is to archive each file separately.

Example

Init state

src/dir/

files in there:

file1.xls, file2.xls

archive/dir

file in there: none

Now compressing files and moving them to another directory:

Final state

src/dir/

file in there: none

archive/dir

files in there:

file1.zip, file2.zip (or .gz)

I don't understand what's is not that clear I am asking?.. Simple question. How to archive file (zip, gzip you name it) before moving it to another directory, so it would appear in that directory archived. — Andrius, Oct 07 '14 at 09:07
So can anyone just put simple code example of that example of mine?:) — Andrius, Oct 07 '14 at 09:14

dom0 · Accepted Answer · 2014-10-07T09:25:01.263

Archives don't really make sense if all you going to do is put one file into each of them.

Or did you mean to compress them, e.g. using the gzip or bz2 module?

If you indeed really want archives with only a single file, create a tar or ZIP object and just add it straight away, e.g. for TarFile.add.

Note that while it is very common when using unix-like operating systems to compress single files using bz2 or gzip doing so is very uncommon on other platforms, e.g. Windows. There the recommendation would be to use ZIP files, even for single files, since they are handled well by applications (Windows Explorer and others).

To put a single file into a ZIP file do something similar to this:

import zipfile
# ...

with zipfile.ZipFile(nameOfOrginalFile + ".zip", "w", zipfile.ZIP_DEFLATED) as zip_file:
    zip_file.write(nameOfOriginalFile)

Not passing ZIP_DEFLATED to ZipFile will result in an uncompressed zip file.

To compress a single file using e.g. gzip:

import gzip

with gzip.GzipFile(nameoforiginalFile + ".gz", "w") as gz:
    with open(nameoforignalfile) as inp_file;
        shutil.copyfileobj(inp_file, gz)

The bz2 and lzma (not available for Python 2) APIs are the same, just import bz2/lzma and use bz2.BZ2File instead.

After both with blocks you can delete the original file (os.remove(file)) and move the archive file to the correct location. Alternatively, create the archive file directly in the correct location (os.path.join the target location and the archive name).

Well I thought archive and compress as synonyms. Well what I meant is not to archive directory, but to yeah, compress files and then move them to another directory. — Andrius, Oct 07 '14 at 09:09
@Andrius: No, they are not. Zip does both, tar only archives and gzip/bzip2/xz only compress. A .tar.gz is combination of tar and gzip. — Jan Hudec, Oct 07 '14 at 09:10
Ok, then I really recommend to go with gzip or bz2 modules. gzip is faster, but compression is worse. bz2 is slower, but compresses better. Usual convention is to name compressed files `.gzip` or `... .bz2`. — dom0, Oct 07 '14 at 09:10
... of course it would really be best to use lzma, but python 2.7 is too old for that module. — Jan Hudec, Oct 07 '14 at 09:12

Jan Hudec · Answer 2 · 2014-10-07T09:15:12.193

The standard library contains zipfile module for working with .zip archives, gzip module for working with .gz compressed files and bz2 module for working with .bz2 compressed files (the later is slower, but yields better compression).

Python 3.3 also introduces [lzma] (for .xz and .lzma files), which has even better compression ratio, but it does not seem to be backported to 2.7.

Note that a single file does not need .tar.gz, a .gz will do. Because .tar.gz is two levels. .tar to put several files together and .gz to compress it and you don't need the first part if you have just one. Zip does both things, so for single file it is slightly less efficient than gz (they use the same compression method), but you may have some tool that understands zip files and not gz files, so there may be some reason to use it.

To create single compressed file with gzip, bz2 or [lzma], you just use open function from the respective module and then use shutil.copyfileobj to copy the content of the source file to the archive.

Python - archiving file before moving to another directory?

2 Answers2