Copy file with pathlib in Python

Question

I try to copy a file with pathlib

import pathlib
import shutil

my_file=pathlib.Path('/etc/hosts')
to_file=pathlib.Path('/tmp/foo')
shutil.copy(my_file, to_file)

I get this exception:

/home/foo_egs_d/bin/python /home/foo_egs_d/src/test-pathlib-copy.py
Traceback (most recent call last):
  File "/home/foo_egs_d/src/test-pathlib-copy.py", line 6, in <module>
    shutil.copy(my_file, to_file)
  File "/usr/lib/python2.7/shutil.py", line 117, in copy
    if os.path.isdir(dst):
  File "/home/foo_egs_d/lib/python2.7/genericpath.py", line 41, in isdir
    st = os.stat(s)
TypeError: coercing to Unicode: need string or buffer, PosixPath found

Process finished with exit code

... how to copy file with pathlib in Python 2.7?

Answer 1

To use shutil.copy:

import pathlib
import shutil

my_file = pathlib.Path('/etc/hosts')
to_file = pathlib.Path('/tmp/foo')

shutil.copy(str(my_file), str(to_file))  # For Python <= 3.7.
shutil.copy(my_file, to_file)  # For Python 3.8+.

The problem is pathlib.Path create a PosixPath object if you're using Unix/Linux, WindowsPath if you're using Microsoft Windows.

With older versions of Python, shutil.copy requires a string as its arguments. For them, use the str function here.

Answer 2

The cause for shutil.copy() not working is that you are not using the latest Python, Python 3.6 shutil.copy() can handle Path objects (or subclasses thereof). That for older versions of Python this throws an error is because those implementations of shutil expect string arguments for copy, and not pathlib.Path type arguments.

What you actually want to be able to write is:

my_file.copy(to_file)

You can subclass Path to include such a method, and adapt the creation of my_file. I find it easier to just graft/monkey-patch/duck-punch it on the existing pathlib.Path

from pathlib import Path


def _copy(self, target):
    import shutil
    assert self.is_file()
    shutil.copy(str(self), str(target))  # str() only there for Python < (3, 6)

Path.copy = _copy

You can put this code anywhere you like, as long as it gets executed before calling the .copy method on any of the Path instances. The argument to .copy() can be a file or a directory.

Answer 3

Since Python 3.5, without importing shutil, you can do:

from pathlib import Path

dest = Path('dest')
src = Path('src')
dest.write_bytes(src.read_bytes()) #for binary files
dest.write_text(src.read_text()) #for text files

For Python 2.7, pathlib2 provides the read_bytes, read_text, write_bytes and write_text methods.

The file will be loaded in memory, so this method is not suitable for files larger than the machines available memory.

As per the comments, one can use write_bytes and read_bytes to copy text files, but if you need to deal with the encoding at copy time write_text an read_text present the advantage of two extra parameters:

• encoding is the name of the encoding used to decode or encode the file
• errors is an optional string that specifies how encoding and decoding errors are to be handled

They both have the same meaning as in open().

Answer 4

How shutil was converted to accept pathlib.Path objects in Python 3.6

As mentioned at in this answer, shutil in Python 3.6 can take pathlib.Path objects.

Since this felt quite magic, I decided to investigate a little bit how it was implemented to see if I would be able to reuse this magic on my own classes.

The improvement was a result of PEP 519.

This generalized a lot of stdlib functionality, and documentation was not consistently updated as a result, including most of shutil which as of 3.7 only documents support in a single function. Welcome to the joys of dynamic typing.

Where documented, the stlib links to the glossary for "path-like objects".

An object representing a file system path. A path-like object is either a str or bytes object representing a path, or an object implementing the os.PathLike protocol. An object that supports the os.PathLike protocol can be converted to a str or bytes file system path by calling the os.fspath() function; os.fsdecode() and os.fsencode() can be used to guarantee a str or bytes result instead, respectively. Introduced by PEP 519.

and that then links to the documentation of os.PathLike:

An abstract base class for objects representing a file system path, e.g. pathlib.PurePath.

New in version 3.6.

abstractmethod __fspath__()

Return the file system path representation of the object.

The method should only return a str or bytes object, with the preference being for str.

The key implementation commits seem to be:

• 3f9183b5aca568867f37c38501fca63911580c66 which basically changes the lowest level path manipulation point at Modules/posixmodule.c
• 568be63248614a2cdd7666a67ddfd16e817f7db9 which adds __fspath__ to pathlib.PurePath

If you want to implement your own path-like classes, you can do it like:

#!/usr/bin/env python3

class MyPath:
    def __init__(self, path):
        self.path = path
    def __fspath__(self):
        return self.path

with open(MyPath('f'), 'w'):
    pass

Tested in Python 3.6.7, Ubuntu 18.10.

Answer 5

You might use pathlib3x - it offers a backport of the latest (at the date of writing this answer Python 3.11.a0) Python pathlib for Python 3.6 or newer, and a few additional functions like copy, copy2, etc ...

$> python -m pip install pathlib3x
$> python
>>> import pathlib3x as pathlib
>>> my_file = pathlib.Path('/etc/hosts')
>>> to_file = pathlib.Path('/tmp/foo')
>>> my_file.copy(to_file)

you can find it on github or PyPi

---

^{Disclaimer: I'm the author of the pathlib3x library.}

Answer 6

You can use pathlib rename method instead of shutil.move().

import pathlib

my_file = pathlib.Path('/etc/hosts')
to_file = pathlib.Path('/tmp/foo')
my_file.rename(to_file)