python os.fdopen(os.open()) can't be used for writing?

Question

This question has to do with the answer to Write file with specific permissions in Python for opening a file for writing (in python) with specific permissions.

The code in the answer looks like:

with os.fdopen(os.open('foo', os.O_APPEND | os.O_CREAT, 0o644)) as out:
  out.write("hello\n")

This code in 2.7.1 (my company does not have 2.7.3 installed) produces:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
IOError: File not open for writing

os.fdopen has its own mode argument, but setting that doesn't help:

>>> with os.fdopen(os.open('foo', os.O_APPEND | os.O_CREAT, 0o644), 'a') as out:
...   out.write("hello\n")
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument

Long story short, I have not been able to figure out how to actually write to a file that has been opened via os.fdopen and os.open. Any ideas? Known bug in 2.7.1?

Thanks in advance!

What happens when you give `O_APPEND | O_WRONLY` to `os.open`? — Fred Foo, Nov 06 '12 at 19:13

pilcrow · Accepted Answer · 2012-11-06T19:30:12.947

8

You must choose one of O_RDONLY, O_WRONLY or O_RDWR as a "basic" mode argument to open().

You did not explicitly do so, so O_RDONLY (zero on many systems) is assumed. Python's os.fdopen sees that you have specified a O_RDONLY and O_APPEND, which is a bit silly. Python complains about this combination with the EINVAL ("Invalid argument") error you see.

(Indeed, if you strace(1) your script — I'm assuming Linux here — I suspect you'll see that no "natural" EINVAL is encountered. Instead, python performs your os.open()/open(2), and then checks flags (F_GETFL) on the file descriptor just before raising the exception.)

edited Nov 06 '12 at 19:30

answered Nov 06 '12 at 19:20

pilcrow

56,591
13
94
135

Thanks! The error messages when using `os.open` with `os.fdopen` or even `os.write` are very vague (invalid argument, bad fd) when the open mode doesn't match the usage. – gowiththeflow Nov 06 '12 at 19:33
@gowiththeflow: those are the C library's error messages for POSIX/system call errors. You can look them up in `errno(3)`. – Fred Foo Nov 06 '12 at 20:27
@larsmans, the OP can look them up with `errno.errorcode[]`, too, but I'd agree with the OP that the error message is somewhat vague. Easy to debug if you already know what the problem is. :) – pilcrow Nov 06 '12 at 20:38

score 0 · Answer 2 · answered Sep 03 '13 at 15:40

0

Very funky indeed.

os.fdopen(os.open("a1", os.O_CREAT | os.O_RDWR | os.O_APPEND | os.O_EXCL))

works, while

os.fdopen(os.open("a1", os.O_CREAT | os.O_WRONLY | os.O_APPEND | os.O_EXCL))

raises an OSError: [Errno 22] Invalid argument to the os.fdopen().

So os.fdopen() needs full read/write access to the FD. Unless you do

os.fdopen(fd, "w")

which than works with write-only files.

answered Sep 03 '13 at 15:40

ddalex

436
4
7

You almost had it.... – Otheus Apr 28 '22 at 09:03

score -1 · Answer 3 · answered Apr 29 '22 at 16:08

Two things:

First, O_APPEND does not make sense with O_CREAT with O_EXCL. It will work if the file does not exist, and fail otherwise. So you can only append to a file that hasn't been created. Drop either O_EXCL or O_APPEND.

Second, python's fdopen will try to open the file with its default read-only mode. That means that in order to open a file-descriptor as a file-handle object for writing or appending (so that you can use the write method or classes that expect that method). Here's how you can do that as an ugly one-liner:

fh=os.fdopen(os.open("a1",os.O_CREAT | os.O_RDWR | os.O_APPEND ),"w")

For clarity:

fd=os.open("a1",os.O_CREAT | os.O_RDWR | os.O_APPEND )
fh=os.fdopen(fd,"w")

In my testing, it did not matter if you used "w" or "a" h/t: ddalex in his answer.

python os.fdopen(os.open()) can't be used for writing?

3 Answers3

Linked