Python - How to copy only new modified files

Question

I am trying to write a python script that will copy certain files from a source folder to a destination, however only newly created files should be copied. For example, the source folder contains 2005 files and the destination folder contains 2000 of them, my script should copy the 5 missing files

In the script below, the logic was to log the modification time of the files with extension ".extension" and trying to get it and compare it in next step

import sys, os, shutil
import glob
import os.path, time

fob     = open(r"C:\Python\Log.txt","a")
dir_src = r"C:\Python\Source"
dir_dst = r"C:\Python\Dest"

for w in os.listdir(dir_src):
    if w.endswith('.extenstion'):
        pathname = os.path.join(dir_src, w)
        if not Date_File in (fob):
            shutil.copy2(pathname, dir_dst)
            fob.write("File Name:   %s" % os.path.basename(pathname))
            fob.write("   Last modified Date:   %s" % time.ctime(os.path.getmtime(pathname)))
            fob.write("   Copied On:   %s" % time.strftime("%c"))
            fob.write("\n")                
fob.close()
os.system("PAUSE")

I could not find a way to compare the dates and check either a file should be copied or not.

Any other approach is welcome.

Have a look at [`filecmp`](https://docs.python.org/2/library/filecmp.html) — Peter Wood, Feb 24 '15 at 14:01

score 1 · Accepted Answer · answered Feb 24 '15 at 13:58

1

You should be able to remove the already existing files of the destination folder from the source folder by doing list(set(os.listdir(dir_src)) - set(os.listdir(dir_dst))) and iterate over this list.

answered Feb 24 '15 at 13:58

Lachezar

6,523
3
33
34

You can iterate over a set. – Padraic Cunningham Feb 24 '15 at 14:04

score 1 · Answer 2 · answered Feb 24 '15 at 14:00

While your comparison file would work, I think in this case a better way would be to compare the getmtime values of both files. Since the files have the same name in both the source and destination directories, you can simple do a getmtime lookup for the same filename in the destination directory.

If the file doesn't exist you should get a os.error, which means you'll have to copy the file anyway.

If the file in the destination directory was modified earlier than the one in the source directory (so getmtime(src_file) > getmtime(dest_file)) you know to copy it as well.

And you can of course still log it if you want to. You could also forgo th entire process and use rsync or something though.

score 0 · Answer 3 · answered Feb 24 '15 at 14:18

What you can do besides this logic is that, You find epoch time(use os.path.getmtime() method) of files in source and destination paths, compare them. If the file in source path has greater epoch value for last modification time then you can copy it in destination path.

Python - How to copy only new modified files

3 Answers3