Python os.remove(filename) throws error "already being used by temp"

Question

My current code is

import os
import pickle
import tempfile

# Pickle on HIGHEST_PROTOCOL breaks on Python 3.6.5
_PICKLE_PROTOCOL = 4
#CHANGED FROM 2 TO 4 TO ACCOMODATE VERSION, DONE BY USER


def _pickle_iterable(filename, iterable):
    with open(filename, 'wb') as pickle_fh:
        pklr = pickle.Pickler(pickle_fh, _PICKLE_PROTOCOL)
        for entry in iterable:
            pklr.dump(entry)
            pklr.clear_memo()


def _open_pickle(filename):
    return open(filename, 'rb')


def _unpickle_iterable(pickle_fh):
    with pickle_fh:
        unpklr = pickle.Unpickler(pickle_fh)
        try:
            while True:
                yield unpklr.load()
        except EOFError:
            pass


def file_buffered_tee(iterable, n=2):
    _,  filename = tempfile.mkstemp()
    try:
        _pickle_iterable(filename, iterable)
        return tuple(_unpickle_iterable(_open_pickle(filename)) for _ in range(n))
    finally:
        os.remove(filename)

os.remove(filename) gives the error

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\s%username%\\AppData\\Local\\Temp\\tmpaoos53ie'

I don't know how to fix it, and the GitHub repo I pulled this from is archived, and I can't open another issue request.

I'm coming back to stackoverflow, and everywhere else I've seen I cannot find an applicable answer, and while I think I understand the code, I keep hitting errors lol

Any help is appreciated! Thank you!

edit 1: forgot imports... dumb mistake

edit 2: I left out code because I thought it was un-needed. Thank you all for being patient :P

edit 3: Traceback:

line37, in file_buffered_tee:

os.remove(filename)

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\%username%\\AppData\\Local\\Temp\\tmp*xxxxxxx*'

edit 4: apparently the same issue was being had here , however it does everything the answer says to do, but returns the same error... Still not answered here and still very confused. Documentation hasnt helped either

This is a Windows-ism. POSIX platforms allow in-use files to be unlinked. — Charles Duffy, Jun 28 '20 at 05:00
Please post the traceback. Then we can see the line that fails. — tdelaney, Jun 28 '20 at 05:00
Your code runs for me without any error. I created a list,`j=[1,2,3,4,5]` and called `file_buffered_tee(j)` — Z4-tier, Jun 28 '20 at 05:01
@CharlesDuffy Does it matter in this case? The context manager should close the file before `_pickle_iterable` returns, right? — Z4-tier, Jun 28 '20 at 05:04
@Z4-tier I dont know what you did, but it still doesnt work for me... I've also tried os.close(filename) before remove, along with os.unlink(filename), where I get the same error — Eternal_Dusk, Jun 28 '20 at 05:23
@Eternal_Dusk Only thing different is I am running it on MacOS. Still, I don't see why it would matter here: you are using a proper context manager to work with the file handler, and it *should* close the file when it goes out of scope. I suppose you could try changing to this: `pickle_fh = open(filename, 'wb')` and then add `pickle_fh.close()` to the end of `_pickle_iterable`, to explicitly release the file. — Z4-tier, Jun 28 '20 at 06:00
@z4-tier, the OP's error _never_ happens on MacOS, because all its filesystems have a proper inode abstraction so they can delete open files and don't try to emulate the semantics of old ones that didn't (as Windows does for backwards compatibility reasons, as some applications decided to use that operating system design flaw as a locking mechanism back in the FAT days). — Charles Duffy, Jun 28 '20 at 17:53
@Z4-tier I changed the code to exactly what you had, yet still ran into an issue. Im wondering whether the code is just incompatible with Windows, which would suck but hey... line 38 returns process cannot access the file, used by another process. — Eternal_Dusk, Jun 29 '20 at 01:37
Ok, so after our experiments involving different ways to close files in python, I think we can be sure that *your* process is doing the right thing and closing the file as expected. I am pretty well convinced that @CharlesDuffy is right: windows is a pig and it's holding that file open outside of your control. This answer (https://stackoverflow.com/a/5422943/6689725) gives a solution that is not quite the same but might be worth a shot. Otherwise: http://ozguru.mu.nu/Photos/2005-11-11--Dilbert_Unix.jpg — Z4-tier, Jun 29 '20 at 02:52
@Z4-tier love that photo lmfao I have another rig running Debian I will try next then, thank you lol — Eternal_Dusk, Jun 29 '20 at 02:54

chasedig · Answer 1 · 2020-06-28T04:48:43.053

0

You can tell tempfile to delete file upon close. This way, when you close the file, the file will be deleted automatically.

This may help you: https://docs.python.org/3/library/tempfile.html

tempfile.NamedTemporaryFile(mode='w+b', buffering=None, encoding=None, newline=None, suffix=None, prefix=None, dir=None, delete=True, *, errors=None)

delete=True parameter is what you're looking for. Now use tempfile to close the file via file.close()

edited Jun 28 '20 at 04:48

answered Jun 28 '20 at 04:46

chasedig

58
3
8

I'm getting the error 'ValueError: not enough values to unpack (expected 2, got 0)' – Eternal_Dusk Jun 28 '20 at 05:25
Mind posting the line giving you the issue? – chasedig Jun 28 '20 at 05:29
3 things I have done: 1) changed tempfile.mkstemp() to tempfile.NamedTemporaryFile(), gave error on line 32 2) added os.close(filename) before os.remove(filename), TypeError: an integer is required (got type str) 3) changed os.close(filename) to filename.close(), AttributeError: 'str' object has no attribute 'close' – Eternal_Dusk Jun 28 '20 at 05:34
1

You can fix 'str' object has no attribute 'close' by instead of supplying the filename, supplying the actual file. This generally looks like: `file = file.open()` but it may be different for tempfile – chasedig Jun 28 '20 at 05:36

Python os.remove(filename) throws error "already being used by temp"

1 Answers1