PermissionError: [WinError 32] for creating files

Question

There is a code segment as follows

try:
    f = h5py.File(filename, 'w-')
except: 
    os.remove(filename)
    f = h5py.File(filename, 'w-')

Running the program getting the error related to the above code segment. I think it is because the file was not closed. Googling the similar error message, it seems that it can be solved using "with" statement, but I am not sure how to modify the above code segment.

OSError                                   Traceback (most recent call last)
<ipython-input-19-1f1f9c5eb3dd> in vid_to_hdf(En, start, end, chunk)
      9     try:
---> 10         f = h5py.File(filename, 'w-')
     11     except:

~\AppData\Local\Continuum\anaconda3\envs\fastai-py37\lib\site-packages\h5py\_hl\files.py in __init__(self, name, mode, driver, libver, userblock_size, swmr, rdcc_nslots, rdcc_nbytes, rdcc_w0, track_order, **kwds)
    407                                fapl, fcpl=make_fcpl(track_order=track_order),
--> 408                                swmr=swmr)
    409 

~\AppData\Local\Continuum\anaconda3\envs\fastai-py37\lib\site-packages\h5py\_hl\files.py in make_fid(name, mode, userblock_size, fapl, fcpl, swmr)
    176     elif mode in ['w-', 'x']:
--> 177         fid = h5f.create(name, h5f.ACC_EXCL, fapl=fapl, fcpl=fcpl)
    178     elif mode == 'w':

h5py\_objects.pyx in h5py._objects.with_phil.wrapper()

h5py\_objects.pyx in h5py._objects.with_phil.wrapper()

h5py\h5f.pyx in h5py.h5f.create()

OSError: Unable to create file (file exists)

During handling of the above exception, another exception occurred:

PermissionError                           Traceback (most recent call last)
<timed eval> in <module>

<ipython-input-19-1f1f9c5eb3dd> in vid_to_hdf(En, start, end, chunk)
     10         f = h5py.File(filename, 'w-')
     11     except:
---> 12         os.remove(filename)
     13         f = h5py.File(filename, 'w-')
     14     # Create dataset within file

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'E23.hdf5'

farzad · Accepted Answer · 2020-08-14T06:57:54.910

As a side note, something is not clear in the shared code sample:

try:
    f = h5py.File(filename, 'w-')
except: 
    os.remove(filename)
    f = h5py.File(filename, 'w-')  # <- why the error handling code is repeating exactly as the code that just failed to run?

When a system resource (like a file) is opened, there are information/state about that resource stored which should be released/closed as a clean behavior of the program (for example closing the file handles of the operating system).

So in Python

my_file = open('myfile.txt', 'r')  # getting the resource
my_file.readlines()  # using the resources
my_file.close()  # closing the file, releasing system resources

To do this easier, some APIs provide a context manager that can be used in a with block:

with open('myfile.txt', 'r') as my_file:
   my_file.readlines()

# after the with block, the context manager automatically calls close() when exiting the runtime context

h5py provides such an API, so the h5py.File instance can be used in a with block as well:

with h5py.File(filename, 'w-') as f:
    pass # do your processing with f

# now after the block, the file is closed

Please note that closing a file does not mean removing it. So after the file is closed in the code and system resources are released, the file stays on disk, until is removed. But in many cases this is the expected behavior, as the file is a result of the program.

If you want to make sure the file is always removed, you can use a try/finally block, but note that then there will be no file remained on disk after your program runs:

try:
   with f = h5py.File(filename, 'w-'):
       pass # use the file resource
finally:
   if os.path.exists(filename):
       os.remove(filename)

Thank you for the reply, may I know what exactly is the difference between "w" and "w-" in the h5py.file()? — user297850, Aug 14 '20 at 17:28
I've never used `h5py`, and referred to the docs for this question. As it's documented "w" truncates an existing files (overwriting the content), but "w-" fails if the file exists. Reading documentations is quite helpful. https://docs.h5py.org/en/latest/high/file.html#opening-creating-files — farzad, Aug 14 '20 at 17:34

score 0 · Answer 2 · answered Aug 14 '20 at 06:38

First things first, try deleting the file before using it.
If you can't do that, try reading the contents of the file, deleting the File object, deleting the file then writing the file.

If that doesn't work, try this:

with h5py.File(filename, 'w-') as f:
    # This is where you put what you're going to do with f
# At the end of the with statement, the object gets deleted.
try:
    os.remove(filename)
except:
    pass

PermissionError: [WinError 32] for creating files

2 Answers2