30

I am attempting to use the 'tempfile' module for manipulating and creating text files. Once the file is ready I want to save it to disk. I thought it would be as simple as using 'shutil.copy'. However, I get a 'permission denied' IOError:

>>> import tempfile, shutil
>>> f = tempfile.TemporaryFile(mode ='w+t')
>>> f.write('foo')
>>> shutil.copy(f.name, 'bar.txt')

Traceback (most recent call last):
  File "<pyshell#5>", line 1, in <module>
    shutil.copy(f.name, 'bar.txt')
  File "C:\Python25\lib\shutil.py", line 80, in copy
    copyfile(src, dst)
  File "C:\Python25\lib\shutil.py", line 46, in copyfile
    fsrc = open(src, 'rb')
IOError: [Errno 13] Permission denied: 'c:\\docume~1\\me\\locals~1\\temp\\tmpvqq3go'
>>>

Is this not intended when using the 'tempfile' library? Is there a better way to do this? (Maybe I am overlooking something very trivial)

sanxiyn
  • 3,648
  • 1
  • 19
  • 15
Ray
  • 187,153
  • 97
  • 222
  • 204

4 Answers4

45

hop is right, and dF. is incorrect on why the error occurs.

Since you haven't called f.close() yet, the file is not removed.

The doc for NamedTemporaryFile says:

Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).

And for TemporaryFile:

Under Unix, the directory entry for the file is removed immediately after the file is created. Other platforms do not support this; your code should not rely on a temporary file created using this function having or not having a visible name in the file system.

Therefore, to persist a temporary file (on Windows), you can do the following:

import tempfile, shutil
f = tempfile.NamedTemporaryFile(mode='w+t', delete=False)
f.write('foo')
file_name = f.name
f.close()
shutil.copy(file_name, 'bar.txt')
os.remove(file_name)

The solution Hans Sjunnesson provided is also off, because copyfileobj only copies from file-like object to file-like object, not file name:

shutil.copyfileobj(fsrc, fdst[, length])

Copy the contents of the file-like object fsrc to the file-like object fdst. The integer length, if given, is the buffer size. In particular, a negative length value means to copy the data without looping over the source data in chunks; by default the data is read in chunks to avoid uncontrolled memory consumption. Note that if the current file position of the fsrc object is not 0, only the contents from the current file position to the end of the file will be copied.

Cory Klein
  • 51,188
  • 43
  • 183
  • 243
K Z
  • 29,661
  • 8
  • 73
  • 78
  • 1
    @jedierikb `os.remove` will work -- or you can just close the `NamedTemporaryFile` after "all done" without `delete=False`. – K Z Jun 23 '13 at 03:06
  • 3
    @KayZhu except that without `delete=False`, `shutil.copy` won't work, resulting in the original `IOError` that started this thread! so in this case I agree with @jedierikb, you need the `os.remove`. – cod3monk3y Nov 14 '14 at 16:15
  • Additionally, the object returned from `NamedTemporaryFile` is a file-like object, so it *could* be used with `shutil.copyfileobj`. – cod3monk3y Nov 14 '14 at 17:07
23

The file you create with TemporaryFile or NamedTemporaryFile is automatically removed when it's closed, which is why you get an error. If you don't want this, you can use mkstemp instead (see the docs for tempfile).

>>> import tempfile, shutil, os
>>> fd, path = tempfile.mkstemp()
>>> os.write(fd, 'foo')
>>> os.close(fd)
>>> shutil.copy(path, 'bar.txt')
>>> os.remove(path)
Paolo Stefan
  • 10,112
  • 5
  • 45
  • 64
dF.
  • 74,139
  • 30
  • 130
  • 136
  • 3
    i think the error comes from the fact that you cannot access the file a second time while it is still open (this is only so on windows, according to the documentation) –  Nov 10 '08 at 01:03
  • 6
    Kay Z has a correct answer below. Plus, s/he explains it better. – Dave Abrahams Mar 10 '12 at 02:01
  • Thanks. This answer is insightful because it gives me to know how to make a temporary directory without it being deleted when the with/as statement is over, seeing as `mkdtemp` works a lot like `mkstemp`. – Brōtsyorfuzthrāx Oct 14 '14 at 07:46
  • 1
    `TemporaryFile` opens the file handle, and `shutil` tries to open the file a second time for writing, which [does not work on Windows](http://stackoverflow.com/a/23212515/1174169), as @hop points out. – cod3monk3y Nov 14 '14 at 16:11
  • This is not the reason, can you edit your answer to be correct? – smci Jul 20 '17 at 09:00
14

Starting from python 2.6 you can also use NamedTemporaryFile with the delete= option set to False. This way the temporary file will be accessible, even after you close it.

Note that on Windows (NT and later) you cannot access the file a second time while it is still open. You have to close it before you can copy it. This is not true on Unix systems.

6

You could always use shutil.copyfileobj, in your example:

new_file = open('bar.txt', 'rw')
shutil.copyfileobj(f, new_file)
Exelian
  • 5,749
  • 1
  • 30
  • 49
Hans Sjunnesson
  • 21,745
  • 17
  • 54
  • 63
  • I think this is the correct answer, but one remark: you probably have to add 'f.seek(0)' first. – squanto773 Aug 18 '18 at 03:24
  • Can you please elaborate on how the 'rw' mode is different from 'r+' and 'w+'? Actualy I can't use it, I get a `ValueError` with message that it `must have exactly one of create/read/write/append mode`... – Jyrkka Apr 25 '20 at 23:06
  • Jyrrka: Wow, this is a blast from the past! If you check out the documentation for `open()`: https://docs.python.org/3/library/functions.html#open Then you see that modes differ in whether they truncate the file or not. – Hans Sjunnesson Apr 29 '20 at 17:11
  • yep, `rw` seems like an invalid mode in Python 3, what was your intention with `rw`? (would not `w` be enough?) – jave.web Apr 16 '21 at 07:26
  • @HansSjunnesson yep, rw seems like an invalid mode in Python 3, what was your intention with rw? would not some w version be enough? **`wb`** worked for me :) – jave.web Apr 16 '21 at 07:33