Copy files using python with complete folder structure

Question

I am switching my SSD with a better one in a few days and I have a bunch of data stored on it that I might regret if deleted. The only type of files I need are PDF files, docx files, txt files and other things. So, I wrote a script to locate those files using python.

# to copy all of my documents into another location.
import sys
import os
import time
import pathlib
import json


filePath=["D:\\", "C:\\Users"]
# ext=['mkv','docx','doc','pdf','mp4','zip',]
fileExt=["**\*.docx","**\*.doc","**\*.pdf"]
fileList={}
for each_drive in filePath:
    fileList[each_drive]={}
    for each_type in fileExt:
        fileList[each_drive][each_type]=list(pathlib.Path(each_drive).glob(each_type))

file1 = open('test.txt', 'w')
for each in fileList.values():
    for each2 in each.values():
        for entry in each2:
            print(entry)
            file1.writelines(str(str(entry)+ "\n"))


file1.close()

This script just locates the file with formats matching the FileExt list and writes out those locations to test.txt file. Now I need to transfer these files while keeping the exact directory structure. For example, If there is a file as

C:\Users\<MyUser>\AppData\Local\Files\S0\1\Attachments\hpe[4].docx

The script should copy the entire directory structure as

<BackupDrive>:\<BackupFolderName>\C\Users\<MyUser>\AppData\Local\Files\S0\1\Attachments\hpe[4].docx

How do I copy using this exact structure.
TLDR: Need to copy files while keeping the directory structure as is using Python
P.S. I am using Windows, with Python 3.8

https://stackoverflow.com/questions/6004073/how-can-i-create-directories-recursively — FloLie, Nov 29 '20 at 16:09
I thank what you shoud do is, to extract all the paths (without file) change the base from C: to BachupDrive: and then use `makedirs` to create the paths. It does so recursively as shown in above post — FloLie, Nov 29 '20 at 16:10

fischmalte · Answer 1 · 2020-11-29T16:22:48.770

0

Since you are able to write data to a file, I am assuming you also know how to read data from that file. Then for each line (say call it source in that file, use shutil.copyfile(source, dest).

You can create the dest string by manipulating source:

# remove 'C:'
str_split = source[2:]

# add backup drive and folder
dest = ''.join(['<BackupDrive>:\<BackupFolderName>', str_split])

As mentioned in the comments, the destination path will not be created automatically, but can be handled like explained here: create destination path for shutil.copy files

edited Nov 29 '20 at 16:22

answered Nov 29 '20 at 16:12

fischmalte

81
7

This does not create the folders on the way if I am correct, does it? – FloLie Nov 29 '20 at 16:16
1

Nope. It does not. https://stackoverflow.com/questions/6004073/how-can-i-create-directories-recursively Following this link and this answer, I need to first create destination directories then move these files? – CrYbAbY Nov 29 '20 at 16:16
I would include this to the answer: https://stackoverflow.com/questions/2793789/create-destination-path-for-shutil-copy-files – FloLie Nov 29 '20 at 16:16

score 0 · Accepted Answer · answered Nov 29 '20 at 16:20

For each line in your file list, do the following:

for filePath in fileList:
    destination = .join(['<BackupDrive>:\<BackupFolderName>', filePath[2:]])
    os.makedirs(os.path.dirname(filePath), exist_ok=True)
    shutil.copy(filePath , destination)

score 0 · Answer 3 · answered Nov 29 '20 at 16:54

Thanks to @Emmo and @FloLie for their Answers. I just had to use the os.makedirs() function with the exist_ok flag set to true for each file in the list.

This is the code placed immediately after the code in the question.

#######################################
# create destination directory
file1=open ('test.txt', 'r')
text= file1.readlines()
# print(text)
for each in text:
    each=each[:-1]
    destination="BackupDIR-"+each[0]+each[2:]
    os.makedirs(os.path.dirname(destination), exist_ok=True)
    shutil.copy(each,destination)

This makes the entire code look like:

# to copy all of my documents into another location.
import os
import time
import pathlib
import json
import shutil


filePath=["D:\\", "C:\\Users"]
# ext=['mkv','docx','doc','pdf','mp4','zip',]
fileExt=["**\*.docx","**\*.doc","**\*.pdf"]
fileList={}
for each_drive in filePath:
    fileList[each_drive]={}
    for each_type in fileExt:
        fileList[each_drive][each_type]=list(pathlib.Path(each_drive).glob(each_type))

file1 = open('test.txt', 'w')
for each in fileList.values():
    for each2 in each.values():
        for entry in each2:
            print(entry)
            file1.writelines(str(str(entry)+ "\n"))
file1.close()

#######################################
# create destination directory
file1=open ('test.txt', 'r')
text= file1.readlines()
# print(text)
for each in text:
    each=each[:-1]
    destination="BackupDIR-"+each[0]+each[2:]
    os.makedirs(os.path.dirname(destination), exist_ok=True)
    shutil.copy(each,destination)

P.S. This answer is only here for people like me who cant make sense of little snippets out of context sometimes

Copy files using python with complete folder structure

3 Answers3