create destination path for shutil.copy files

Question

If a path such as b/c/ does not exist in ./a/b/c , shutil.copy("./blah.txt", "./a/b/c/blah.txt") will complain that the destination does not exist. What is the best way to create both the destination path and copy the file to this path?

score 126 · Answer 1 · edited Jul 21 '20 at 14:51

To summarize info from the given answers and comments:

For python 3.2+:

os.makedirs before copy with exist_ok=True:

os.makedirs(os.path.dirname(dest_fpath), exist_ok=True)
shutil.copy(src_fpath, dest_fpath)

For python < 3.2:

os.makedirs after catching the IOError and try copying again:

try:
    shutil.copy(src_fpath, dest_fpath)
except IOError as io_err:
    os.makedirs(os.path.dirname(dest_fpath))
    shutil.copy(src_fpath, dest_fpath)

Although you could be more explicit about checking errno and/or checking if path exists before makedirs, IMHO these snippets strike a nice balance between simplicity and functionality.

weird that copying files and creating directories are in two different modules. — user313032, Aug 24 '23 at 22:05

score 42 · Answer 2 · answered May 08 '10 at 11:11

42

Use os.makedirs to create the directory tree.

answered May 08 '10 at 11:11

Philipp

48,066
12
84
109

4

Note that the `exists_ok` option is only present in Python 3.2+ – Grant Humphries Sep 13 '17 at 18:42
15

It's `exist_ok`, not `exists_ok` – CharlesB Dec 07 '17 at 13:53

score 31 · Answer 3 · answered Jul 19 '10 at 19:12

31

I use something similar to this to check if the directory exists before doing things with it.

if not os.path.exists('a/b/c/'):
    os.mkdir('a/b/c')

answered Jul 19 '10 at 19:12

thegrinner

11,546
5
41
64

1

As far as I know, this won't work in Python 2.7: OSError: [Errno 2] No such file or directory: './a/b/c' – Tomas Tomecek Jan 21 '13 at 09:33
4

I prefer to use `os.makedirs` instead, which would create parent directories if they don't exist. – feilong Jan 13 '16 at 18:43
3

Be aware that this suffers from a race condition (if someone else or another thread create the directory between the check and calling `makedirs`). Better to call `os.makedirs` and catch the exception if the folder exist. Check SoF for directory creation. – farmir Apr 27 '16 at 07:40

score 21 · Answer 4 · edited Oct 05 '21 at 18:53

21

This is the EAFP way, which avoids races and unneeded syscalls:

import errno
import os
import shutil

src = "./blah.txt"
dest = "./a/b/c/blah.txt"
# with open(src, 'w'): pass # create the src file
try:
    shutil.copy(src, dest)
except IOError as e:
    # ENOENT(2): file does not exist, raised also on missing dest parent dir
    if e.errno != errno.ENOENT:
        raise
    # try creating parent directories
    os.makedirs(os.path.dirname(dest))
    shutil.copy(src, dest)

edited Oct 05 '21 at 18:53

urig

16,016
26
115
184

answered Sep 02 '17 at 14:08

Mr_and_Mrs_D

32,208
39
178
361

1

There is still a race if `dest` dir is created after the call to `shutil.copy` but before the call to `os.makedirs`. – Lii Jan 04 '18 at 16:28

score 9 · Answer 5 · answered Apr 20 '20 at 16:21

9

For 3.4/3.5+ you can use pathlib:

Path.mkdir(mode=0o777, parents=False, exist_ok=False)

So if there might be multiple directories to create and if they might already exist:

pathlib.Path(dst).mkdir(parents=True, exist_ok=True)

answered Apr 20 '20 at 16:21

johnson

3,729
3
31
32

score 8 · Answer 6 · answered Jun 02 '18 at 08:23

8

How about I use split to get the dir out of the path

dir_name, _ = os.path.split("./a/b/c/blah.txt")

then

os.makedirs(dir_name,exist_ok=True)

and finally

shutil.copy("./blah.txt", "./a/b/c/blah.txt")

answered Jun 02 '18 at 08:23

peterb

891
8
7

Note that if the `dir_name` contains subdirectories that don't exist you'll want to adjust the code to `os.makedirs(dir_name + '/',exist_ok=True)` – datalifenyc Aug 19 '21 at 15:39

score 1 · Answer 7 · answered May 25 '18 at 00:12

1

My five cents there would be is the next approach:

# Absolute destination path.
dst_path = '/a/b/c/blah.txt'
origin_path = './blah.txt'
not os.path.exists(dst_path) or os.makedirs(dst_path)
shutil.copy(origin_path, dst_path)

answered May 25 '18 at 00:12

Andriy Ivaneyko

20,639
6
60
82

James Mishra · Answer 8 · 2021-08-09T01:22:50.193

A lot of the other answers are for older versions of Python, although they may still work, you can handle errors a lot better with newer Pythons.

If you are using Python 3.3 or newer, we can catch FileNotFoundError instead of IOError. We also want to differentiate between the destination path not existing and the source path not existing. We want to swallow the former exception, but not the latter.

Finally, beware that os.makedirs() recursively creates missing directories one at a time--meaning that it is not an atomic operation. You may witness unexpected behavior if you have multiple threads or processes that might try to create the same directory tree at the same time.

def copy_path(*, src, dst, dir_mode=0o777, follow_symlinks: bool = True):
    """
    Copy a source filesystem path to a destination path, creating parent
    directories if they don't exist.

    Args:
        src: The source filesystem path to copy. This must exist on the
            filesystem.

        dst: The destination to copy to. If the parent directories for this
            path do not exist, we will create them.

        dir_mode: The Unix permissions to set for any newly created
            directories.

        follow_symlinks: Whether to follow symlinks during the copy.

    Returns:
        Returns the destination path.
    """
    try:
        return shutil.copy2(src=src, dst=dst, follow_symlinks=follow_symlinks)
    except FileNotFoundError as exc:
        if exc.filename == dst and exc.filename2 is None:
            parent = os.path.dirname(dst)
            os.makedirs(name=parent, mode=dir_mode, exist_ok=True)
            return shutil.copy2(
                src=src,
                dst=dst,
                follow_symlinks=follow_symlinks,
            )
        raise

create destination path for shutil.copy files

8 Answers8

Linked