1

I'm trying to make a script work under windows, but seem to run in circles. It works on posix filesystems. In the code below, i'm using the library tempfile in a with statement to create a tempfile and copy the source into it. Then i want to use the function strip_whitespaces_from_file() on this tempfile. But instead the permission error below. 

import tempfile
import shutil
import fileinput

source_file_name = r"C:\Users\morph\Stuff\test.txt"


def strip_whitespaces_from_file(input_file_path: str):
    with fileinput.FileInput(files=(input_file_path,), inplace=True) as fp:
        for line in fp:
            print(line.strip())


with tempfile.NamedTemporaryFile(delete=False) as dest_file:
    shutil.copy(source_file_name, dest_file.name)
    strip_whitespaces_from_file(dest_file.name)

Instead i get this output:

"PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 
'C:\\Users\\morph\\AppData\\Local\\Temp\\tmpbvywadw7' -> 'C:\\Users\\morph\\AppData\\Local\\Temp\\tmpbvywadw7.bak'

The error message seems quite straightforward, but i can't find a way around it. There are a couple of answers, e.g. this one, but that would mean i have to close the file before i work on it. Apparently the tempfile is opened by the same process that wants to write to it? Isn't that the whole point? I'm confused.

Edit: Below is my clunky workaround. But it illustrates the need for delete=False. If i remove it i get a FileNotFoundError. To me that looks like the file i removed immediately after creating it in the first line of below code. The manual closing and os.unlink also shouldn't be needed.

with tempfile.NamedTemporaryFile(delete=False) as temp:
    with open(source_file_name, 'r') as src_file:
        for line in src_file:
            stripped_line = line.strip()+"\n"
            temp.write(stripped_line.encode('utf-8'))
    src_file.close()
    temp.close()
    # here would be an external converter using the temp file as argument
    os.unlink(temp.name)
    print(os.path.exists(temp.name))
  • 1
    If you don't want to close the file after copying, before doing your `strip_whitespaces`, you could refactor your code to do the whitespace stripping during the copy process.. – Alan Hoover May 30 '20 at 17:24
  • `shutil.copyfileobj(open(source_file_name), dest_file)`? – CristiFati May 30 '20 at 17:27
  • Internally, for an in-place operation, [fileinput](https://github.com/python/cpython/blob/7b78e7f9fd77bb3280ee39fb74b86772a7d46a70/Lib/fileinput.py#L335) needs to call `os.rename(self._filename, self._backupfilename)`. In Windows, renaming requires `DELETE` access; it's essentially an unlink / link operation. Out of the box, Windows Python only supports shared delete access for delete-on-close temporary files. Your existing open from `NamedTemporaryFile` does not allow shared delete/rename access because you passed `delete=False`, i.e. it's not a delete-on-close open. – Eryk Sun May 30 '20 at 18:23
  • BTW, the system error message for `ERROR_SHARING_VIOLATION` (winerror 32) is misleading. This error has nothing at all to do with processes. Windows filesystems keep a share-access record for every file, which counts the number of times a file has been opened with read/execute, write/append, or delete access, and also the number of times said access was shared. If you need delete access, and all previous opens didn't share delete access, then it's a sharing violation. If you are not willing to share delete access, and previous opens have delete access, then it's also a sharing violation. – Eryk Sun May 30 '20 at 18:27
  • Thanks all, could you explain a bit further? Just so we are on the same side. I was going for opening a context manager for a tempfile. In that tempfile i copy the source file, do a line strip with the `strip_whitespaces_from_file` function and then pass it to a external process for further processing. The whole idea of the operation was to avoid issues like you are describing in the first place. –  May 31 '20 at 16:15
  • I think will go with this rather unwieldy explicit manual delete now: ` with tempfile.NamedTemporaryFile(delete=True) as temp: with open(activity_file_name, 'r') as a_file: for line in a_file: stripped_line = line.strip()+"\n" temp.write(stripped_line.encode('utf-8')) a_file.close() temp.close() gpsbabel_convert(temp.name, target_file_name, "tcx") os.unlink(temp.name) print(os.path.exists(temp.name))` –  May 31 '20 at 19:51
  • Since you don't want the file to be opened with the delete-on-close flag, you won't be able to share delete/rename access without going below Python's standard library to call WinAPI `CreateFileW` directly. Barring that, `strip_whitespaces_from_file` has to be refactored to work on the `dest_file` open directly. As suggested by Alan in the first comment, you can integrate that into your own copy routine instead of calling `shutil.copy`. – Eryk Sun May 31 '20 at 19:51
  • Edit your question to show the workaround code clearly. It doesn't make sense to me why you would need to manually delete `temp` given it's opened with `delete=True`. In Windows in particular, `delete=True` tells the OS to automatically delete the file as soon as it's closed, which includes the case of the process terminating/crashing. Even in Unix, with `delete=True`, Python tries to delete the file when it's closed, though it's not guaranteed in case the process is killed or crashes. – Eryk Sun May 31 '20 at 20:23
  • I edited my question with the workaround. It is basically my version of @AlanHoover suggestion to the best of my knowledge. Yet all that seems a bit redundant. –  Jun 01 '20 at 06:46
  • Okay, in the comment you had `delete=True`, but I see you switched back. You can only use `delete=True` in Windows if the converter opens the file with shared delete access. If it doesn't, then opening the file in the converter will fail with a sharing violation. Python itself only shares read and write access normally. It shares delete access only when it requests delete access for a delete-on-close temp file. This means a Python program normally cannot access a file that's opened with delete access. For example, this fails: `f = NamedTemporaryFile();` `open(f.name)`. – Eryk Sun Jun 01 '20 at 08:30
  • Ok got it, so how is this handled normally then? The whole tempfile is somewhat redundant if i cannot use `delete=True ` and have to clean up after the operation. Is that just a fact that isn't supposed to work due to the limitations you described? I can't really see a difference between just creating a new file and then deleting manually. –  Jun 01 '20 at 09:14

0 Answers0