30

I want to replace the contents of a hidden file, so I attempted to open it in w mode so it would be erased/truncated:

>>> import os
>>> ini_path = '.picasa.ini'
>>> os.path.exists(ini_path)
True
>>> os.access(ini_path, os.W_OK)
True
>>> ini_handle = open(ini_path, 'w')

But this resulted in a traceback:

IOError: [Errno 13] Permission denied: '.picasa.ini'

However, I was able to achieve the intended result with r+ mode:

>>> ini_handle = open(ini_path, 'r+')
>>> ini_handle.truncate()
>>> ini_handle.write(ini_new)
>>> ini_handle.close()

Q. What is the difference between the w and r+ modes, such that one has "permission denied" but the other works fine?

UPDATE: I am on win7 x64 using Python 2.6.6, and the target file has its hidden attribute set. When I tried turning off the hidden attribute, w mode succeeds. But when I turn it back on, it fails again.

Q. Why does w mode fail on hidden files? Is this known behaviour?

zedex
  • 311
  • 1
  • 3
  • 7

3 Answers3

41

It's just how the Win32 API works. Under the hood, Python's open function is calling the CreateFile function, and if that fails, it translates the Windows error code into a Python IOError.

The r+ open mode corresponds to a dwAccessMode of GENERIC_READ|GENERIC_WRITE and a dwCreationDisposition of OPEN_EXISTING. The w open mode corresponds to a dwAccessMode of GENERIC_WRITE and a dwCreationDisposition of CREATE_ALWAYS.

If you carefully read the remarks in the CreateFile documentation, it says this:

If CREATE_ALWAYS and FILE_ATTRIBUTE_NORMAL are specified, CreateFile fails and sets the last error to ERROR_ACCESS_DENIED if the file exists and has the FILE_ATTRIBUTE_HIDDEN or FILE_ATTRIBUTE_SYSTEM attribute. To avoid the error, specify the same attributes as the existing file.

So if you were calling CreateFile directly from C code, the solution would be to add in FILE_ATTRIBUTE_HIDDEN to the dwFlagsAndAttributes parameter (instead of just FILE_ATTRIBUTE_NORMAL). However, since there's no option in the Python API to tell it to pass in that flag, you'll just have to work around it by either using a different open mode or making the file non-hidden.

Adam Rosenfield
  • 390,455
  • 97
  • 512
  • 589
  • +1 for linking Win32 API docs. Your explanation is exactly what I was looking for. By myself, I only got as far as the [implementation](http://hg.python.org/cpython/file/c6880edaf6f3/Objects/fileobject.c#l318) of Python's `open` function. – zedex Nov 04 '12 at 05:16
  • @MrGamgee: Yeah, Python calls the `_wfopen` function, which is part of the Microsoft C runtime library (CRT). If you have Visual Studio installed, you can look at the CRT source, typically in somewhere like `C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\crt\src`. The implementation of `_wfopen` eventually calls `CreateFile` (after a few intermediate function calls). – Adam Rosenfield Nov 05 '12 at 01:06
  • @Adam, how do you trace what underlying C function gets called? – Minh Tran Apr 19 '18 at 00:11
  • I cannot believe this isn't more remarked upon. THANKS GOOGLE! Just busted my ass for over an hour assuming something was up with the file permissions because I couldn't amend or write to a hidden file. In all my years of coding I'd never come across this. This seems to be the only place according to google where people are discussing this behaviour. – hedgehog90 Apr 23 '18 at 12:50
4

Here are the detailed differences:-

``r'' Open text file for reading. The stream is positioned at the beginning of the file.

``r+'' Open for reading and writing. The stream is positioned at the beginning of the file.

``w'' Truncate file to zero length or create text file for writing. The stream is positioned at the beginning of the file.

``w+'' Open for reading and writing. The file is created if it does not exist, otherwise it is truncated. The stream is positioned at the beginning of the file.

``a'' Open for writing. The file is created if it does not exist. The stream is positioned at the end of the file. Subsequent writes to the file will always end up at the then current end of file, irrespective of any intervening fseek(3) or similar.

``a+'' Open for reading and writing. The file is created if it does not exist. The stream is positioned at the end of the file. Subse- quent writes to the file will always end up at the then current end of file, irrespective of any intervening fseek(3) or similar.

From python documentation - http://docs.python.org/2/tutorial/inputoutput.html#reading-and-writing-files:-

On Windows, 'b' appended to the mode opens the file in binary mode, so there are also modes like 'rb', 'wb', and 'r+b'. Python on Windows makes a distinction between text and binary files; the end-of-line characters in text files are automatically altered slightly when data is read or written. This behind-the-scenes modification to file data is fine for ASCII text files, but it’ll corrupt binary data like that in JPEG or EXE files. Be very careful to use binary mode when reading and writing such files. On Unix, it doesn’t hurt to append a 'b' to the mode, so you can use it platform-independently for all binary files.

So if you are using w mode, you are actually trying to create a file and you may not have the permissions to do it. r+ is the appropriate choice.

If you are in a situation where you do not yet know where your .picasi.ini exists or not and your windows user has file creation permissions in that directory and you want to append new information instead of starting at the beginning of the file (a.k.a "append"), then a+ will be the appropriate choice.

It has nothing to do with whether your file is hidden or not.

Calvin Cheng
  • 35,640
  • 39
  • 116
  • 167
  • -1 for copying and pasting from Python docs and not reading the question. I used `r+` mode which allows updating, ie both reading and writing. – zedex Nov 04 '12 at 03:21
  • You say "`w` Truncate file to zero length or create text file for writing". But then you say later "if you are using `w` mode, you are actually trying to create a file" Which is it? – zedex Nov 04 '12 at 03:37
  • If it doesn't exist, it creates the file. If it exists, it truncates the file to zero length so anything you write on it begins from the start of the file. – Calvin Cheng Nov 04 '12 at 03:39
  • You seem to be missing my point. The reason I checked whether the file exists and whether I have write permissions was merely to rule out possible sources of the IOError for those reading the question, not because I don't know whether the file exists! – zedex Nov 04 '12 at 03:39
  • 3
    "It has nothing to do with whether your file is hidden or not." Please consider this fact: (1) Set hidden attribute. (2) `w` mode fails. (3) Unset hidden attribute (4) `w` mode succeeds. This is repeatable. I have just checked with `ini` and `jpg` files. – zedex Nov 04 '12 at 03:43
  • This answer is confidently wrong, I wish you would just delete it. – oblio Feb 01 '23 at 10:57
3

Thanks for this thread; I had the same issue today. My workaround is as follows. Works with Python 3.7

import os

GuiPanelDefaultsFileName = 'panelDefaults.json'
GuiPanelValues = {
    '-FileName-'      : os.getcwd() + '\\_AcMovement.xlsx',
    '-DraftEmail-'    : True,
    '-MonthComboBox-' : 'Jun',
    '-YearComboBox-'  : '2020'
}

# Unhide the file via OS
if os.path.isfile(GuiPanelDefaultsFileName):
    os.system(f'attrib -h {GuiPanelDefaultsFileName}')

# Write dict values to json
with open(GuiPanelDefaultsFileName, 'w') as fp:
    json.dump(GuiPanelValues, fp, indent=4)

# Make it hidden again
os.system(f'attrib +h {GuiPanelDefaultsFileName}')
Juno55
  • 31
  • 4