How to skip existing files in sub folders and copy only new files

Question

I am copy folder and all sub folders inside the main folder using shutil copytree

import shutil
import sys
import os
import re

SOURCE_FOLDER = sys.argv[1]
DESTINATION_FOLDER = sys.argv[2]


def copyDirectory(SOURCE_FOLDER, DESTINATION_FOLDER):
    try:
        print SOURCE_FOLDER
        print DESTINATION_FOLDER
        shutil.copytree(SOURCE_FOLDER, DESTINATION_FOLDER)
    # Directories are the same
    #except:
    #   print "Not copied"
    except shutil.Error as e:
        print('Directory not copied. Error: %s' % e)
    # Any error saying that the directory doesn't exist
    except OSError as e:
        print('Directory not copied. Error: %s' % e)

copyDirectory(SOURCE_FOLDER,DESTINATION_FOLDER)

The problem is if the directory exists it throws error

Directory not copied. Error: [Errno 17] File exists: 'destination'

What i want is if directory already exists it want to check all the sub directories and if sub directory also exists it should check all the files in it and it should skip the existing files and copy the new files in that sub directory,If sub direscotry not exists then it should copy that sub directory

Note: Sub directories might be nested(Sub directory of sub directory).

But the above script is not working what should i add to that script?

From the docs _"The destination directory must not already exist."_ Perhaps you could copy the code and tweak as needed. — tdelaney, Mar 14 '16 at 05:51

score 1 · Answer 1 · answered Mar 14 '16 at 06:16

1

shutil.copytree isn't written to skip existing destination files and directories. From the docs

The destination directory must not already exist.

You will need to write your own solution. The existing copytree code is a good start.

answered Mar 14 '16 at 06:16

tdelaney

73,364
6
83
116

Yaron · Answer 2 · 2016-03-14T07:43:58.053

0

In order to check if directory is already exists you can use: os.path.exists(directory)

if not os.path.exists(DESTINATION_FOLDER):
    shutil.copytree(SOURCE_FOLDER, DESTINATION_FOLDER)

If the dest directory already exists you can get run your functions on the sub-directories of the src-dir. You can get a list of all src-dir sub-directories using the following function which get directory name as input, and return a list of sub-directories

def SubDirPath (d):
    return filter(os.path.isdir, [os.path.join(d,f) for f in os.listdir(d)])

using this list of directories you can execute your function again, on each instance of the directory.

For each directory which exists in both : src and dst - you'll need to check for every file in src-dir if the file also exists in the dst-dir.

Best Regards,

Yaron

edited Mar 14 '16 at 07:43

answered Mar 14 '16 at 06:54

Yaron

10,166
9
45
65

it will be good if you give me full working code the above code confuses me? – Mar 14 '16 at 07:04
also what is d and f – Mar 14 '16 at 07:04
I've wrapped the filter command, with a function which receives directory as input, and return a list of sub-directories for that directory – Yaron Mar 14 '16 at 07:45

score 0 · Answer 3 · answered Jan 26 '22 at 19:25

0

With python3, you can use shutil.copytree with an option to ignore existing dirs error:

shutil.copytree(SOURCE_FOLDER, DESTINATION_FOLDER, dirs_exist_ok=True)

answered Jan 26 '22 at 19:25

Eugene Dudnyk

5,553
1
23
48

How to skip existing files in sub folders and copy only new files

3 Answers3