Opening a file in an editor while it's open in a script

Question

I have the following code:

import os
import sys
import tempfile
import subprocess

with tempfile.NamedTemporaryFile('w+') as f:
    if sys.platform == 'linux':
        subprocess.call('vim', f.name)
    elif sys.platform == 'nt':
        os.system(f.name)

It opens foobar.txt using either vim on Linux, or the default editor on Windows. On Linux it works fine: tempfile.NamedTemporaryFile() creates a temporary file and vim opens it. On Windows, however, the system says:

The process cannot access the file because it is being used by another process.

I guess that's because the script is currently using the file.

Why does it work on Linux, and how do I get it to work on Windows?

This is basic Windows I/O. All files are opened with a particular access (read/write data, delete, set attributes, etc) and an access sharing mode. `NamedTemporary` opens the file with delete access, as required by the delete-on-close flag, and shares all access (read, write, and delete). Subsequently opening the file again requires sharing delete access, which most programs don't allow. — Eryk Sun, Jan 07 '16 at 09:47
FYI, you can undo the effect of the delete-on-close flag by opening a 2nd handle for the file before closing the first. When the first handle is closed, it sets a delete disposition on the file, but it isn't deleted until all handles are closed. Use the 2nd handle to undo the delete disposition via `SetFileInformationByHandle`. Now you can close the 2nd handle, and the file won't be deleted. — Eryk Sun, Jan 07 '16 at 09:51
@eryksun: Ah, thanks. But Unix allows multiple processes to write to the same file as PM 2Ring said right? — Remi Guan, Jan 07 '16 at 09:54
So does Windows. The default sharing mode used by most programs is to share read and write access. The issue here is that the file in this case is opened with delete access, so opening it again requires sharing delete access. — Eryk Sun, Jan 07 '16 at 09:55
@eryksun: Well, I understand now. So I think you could post an answer. :) *Well, I'm thinking about PM 2Ring's comments which blow christutty's answer. They're also useful for me. A community wiki?* — Remi Guan, Jan 07 '16 at 09:57
Haha, @KevinGuan Knowing eryksun from a long time, he has solved more problems in comments than answers, one reason why I like him as a person. :) (Hopefully he will post an answer this time) — Bhargav Rao, Jan 07 '16 at 10:39
@KevinGuan: If you close a `tempfile.NamedTemporaryFile` (or a plain `tempfile.TemporaryFile``) it will be destroyed. See [the docs](https://docs.python.org/3/library/tempfile.html) for details. Perhaps you could use a `tempfile.mkstemp` file? — PM 2Ring, Jan 07 '16 at 12:39
@KevinGuan: Unix allows multiple processes to write to the same file, but you do have to be careful, as discussed in [this U&L question](http://stackoverflow.com/questions/7842511/safe-to-have-multiple-processes-writing-to-the-same-file-at-the-same-time-cent). — PM 2Ring, Jan 07 '16 at 12:40
@PM2Ring: Thanks for the comments move, think about post an answer? :) — Remi Guan, Jan 07 '16 at 12:41
I don't have a general solution. For example, notepad wants exclusive write access when saving, i.e. when notepad saves the file it calls `CreateFile` with only read sharing, which fails if the file is open with write access in another program. The error you're actually seeing is from cmd.exe, which first tries to call `CreateProcess` on the file. That dies on a sharing violation since a process image is mapped without write sharing. You may as well use `mkstemp`, as suggested, since that's much simpler than undoing the effect of `FILE_FLAG_DELETE_ON_CLOSE` to manually close the file. — Eryk Sun, Jan 07 '16 at 12:49
@KevinGuan: Really, it's more an OS behaviour question than a programming question, so it's not exactly on-topic for SO, IMHO. And the *nix experts over on U&L know _far_ more about the details of this stuff than I do. — PM 2Ring, Jan 07 '16 at 12:50
@eryksun: Yeah, I just want to know *why and how to solve this*. `mkstemp` is a good idea in this case since *Unix's mode isn't safe*. And your explanation is good and I understand now. So I think these would be enough to be an answer. — Remi Guan, Jan 07 '16 at 12:51
@PM2Ring, it's not hard to undo the effect of `FILE_FLAG_DELETE_ON_CLOSE`. It would just be easier to use `mkstemp` from the start if the intention is to first close the file and manually delete it. — Eryk Sun, Jan 07 '16 at 12:53
@PM2Ring: Well, this question is also about Windows I/O, I think just move this question to U&L maybe isn't really good. However, yours comments and eryksun's are already solved my problem. — Remi Guan, Jan 07 '16 at 12:54
@KevinGuan: Cool. And you probably wouldn't _need_ to write a question on U&L, just search the existing questions. I don't know where you'd ask a similar Windows-based question... — PM 2Ring, Jan 07 '16 at 12:58

score 0 · Answer 1 · answered Jun 17 '17 at 08:49

I've run into this problem before. My problem was that I had to write to a file and then use that file's name as an argument in a command.

The reason this works in Linux is that, as @PM 2Ring said in the comments, Linux allows multiple processes to write to the same file, but Windows does not.

There are two approaches to tackle this.

One is to create a temporary directory and create a file in that directory.

# Python 2 and 3
import os
import tempfile

temp_dir = tempfile.mkdtemp()
try:
    temp_file = os.path.join(temp_dir, 'file.txt')
    with open(temp_file, 'w') as f:
        pass  # Create the file, or optionally write to it.
    try:
        do_stuff(temp_file)  # In this case, open the file in an editor.
    finally:
        os.remove(file_name)
finally:
    os.rmdir(temp_dir)

# Python 3 only
import tempfile

with tempfile.TemporaryDirectory() as temp_dir:
    temp_file = os.path.join(temp_dir, 'file.txt')
    with open(temp_file, 'w') as f:
        pass  # Create the file, or optionally write to it.
    do_stuff(temp_file)
    # with tempfile.TemporaryDirectory(): automatically deletes temp_file

Another approach is to create the temporary file with delete=False so that when you close it, it isn't deleted, and then delete it manually later.

# Python 2 and 3
import os
import tempfile

fp = tempfile.NamedTemporaryFile(suffix='.txt', delete=False)
try:
    fp.close()
    do_stuff(fp.name)
finally:
    os.remove(fp.name)

Here is a little context manager that can make files:

import os
import tempfile

_text_type = type(u'')

class ClosedTemporaryFile(object):
    __slots__ = ('name',)
    def __init__(self, data=b'', suffix='', prefix='tmp', dir=None):
        fp = tempfile.mkstemp(suffix, prefix, dir, isinstance(data, _text_type))
        self.name = fp.name
        if data:
            try:
                fp.write(data)
            except:
                fp.close()
                self.delete()
                raise
        fp.close()

    def exists(self):
        return os.path.isfile(self.name)

    def delete(self):
        try:
            os.remove(self.name)
        except OSError:
            pass

    def open(self, *args, **kwargs):
        return open(self.name, *args, **kwargs)

    def __enter__(self):
        return self.name

    def __exit__(self, exc_type, exc_val, exc_tb):
        self.delete()

    def __del__(self):
        self.delete()

Usage:

with ClosedTemporaryFile(suffix='.txt') as temp_file:
    do_stuff(temp_file)

Opening a file in an editor while it's open in a script

1 Answers1