Permission error cloning git repository in temporary folder

Question

Code in question can explain more than I can say in text I think. I've simplified it massively to keep it readable but in essence this is what I'm running.

def main():
    with tempfile.TemporaryDirectory() as td:
        for repo in repos:
            subprocess.run("git clone --mirror {} {}".format(os.path.join(td, repo.path), repo.url)

The cloning part works just fine and goes through the entire list sucessfully. What doesn't is that when the "with ... as ..." exits it throws the following error(traceback starts at __exit__ of the context manager):

    os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\USERNAME\\AppData\\Local\\Temp\\tmp4ew2qffb\\sources\\REPONAME\\objects\\pack\\pack-abd0ff87083dbbcb90f707d8f2d53c730583bb6e.idx'

Running the script as admin doesn't help either. What is going wrong here?

EDIT: I've dug into it and it turns out python 3.7 the TemporaryDirectory cleanup does not support cleaning up read-only files on Windows.

If that edit concludes your question, theb Instead of doing edit you should post this as answer and mark accepyed — Marcin Orlowski, Apr 24 '20 at 15:29

score 0 · Accepted Answer · answered Jun 10 '20 at 06:45

Normally adding at the end of your with tempfile... usage, so before leaving the context something like

for fname in pathlib.Path(td).glob('**/*.*'):  # make all writable for deletion
    fname.chmod(stat.S_IWRITE)

should help.

Note, occasionally I still see PermissionError ("still in use"), not sure yet this is something special in my environment.

score 0 · Answer 2 · answered Jan 07 '23 at 11:06

This is related but not exactly the same as Deleting read-only directory in Python – one possible work-around from this answer is to explicitly fix the permissions before exiting the TemporaryDirectory() context manager. See akaihola/darker#453 for details. Here's a copy of the implementation:

import os
import sys
from pathlib import Path
from typing import Union

WINDOWS = sys.platform.startswith("win")

def fix_py37_win_tempdir_permissions(dirpath: Union[str, Path]) -> None:
    """Work around a `tempfile` clean-up issue on Windows with Python 3.7

    Call this before exiting a ``with TemporaryDirectory():`` block or in teardown for
    a Pytest fixture which creates a temporary directory.

    See discussion in https://github.com/akaihola/darker/pull/393
    Solution borrowed from https://github.com/python/cpython/pull/10320

    :param dirpath: The root path of the temporary directory

    """
    if not WINDOWS or sys.version_info >= (3, 8):
        return
    for root, dirs, files in os.walk(dirpath):
        for name in dirs + files:
            path = os.path.join(root, name)
            try:
                os.chflags(path, 0)  # type: ignore[attr-defined]
            except AttributeError:
                pass
            os.chmod(path, 0o700)

and here's how to use it in Pytest unit tests or when creating a temporary directory using tempfile:

import pytest

from my_utils import fix_py37_win_tempdir_permissions

@pytest.fixture
def myfixture(tmp_path):
    # setup code here
    yield tmp_path
    fix_py37_win_tempdir_permissions(tmp_path)


def myfunc():
    with TemporaryDirectory() as tmpdir:
        # work on the temporary directory here
        fix_py37_win_tempdir_permissions(tmp_path)

Permission error cloning git repository in temporary folder

2 Answers2