Python ctypes: Python file object <-> C FILE *

Question

I am using ctypes to wrap a C-library (which I have control over) with Python. I want to wrap a C-function with declaration:

int fread_int( FILE * stream );

Now; I would like to open file in python, and then use the Python file-object (in some way??) to get access to the underlying FILE * object and pass that to the C-function:

# Python
fileH = open( file , "r")
value = ctypes_function_fread_int( ????? )
fileH.close()

Is the Python file <-> FILE * mapping at all possible?

Joakim

score 29 · Answer 1 · answered Sep 25 '10 at 15:33

A Python file object does not necessarily have an underlying C-level FILE * -- at least, not unless you're willing to tie your code to extremely specific Python versions and platforms.

What I would recommend instead is using the Python file object's fileno to get a file descriptor, then use ctypes to call the C runtime library's fdopen to make a FILE * out of that. This is a very portable approach (and you can go the other way, too). The big issue is that buffering will be separate for the two objects opened on the same file descriptor (the Python file object, and the C FILE *), so be sure to flush said objects as often as needed (or, open both as unbuffered, if that's more convenient and compatible with your intended use).

Thanks a lot -I had this feeling that the C-level FILE * would not be very robust. I did not know about the fileno attribute of the python file object - but that seems like a natural way to go. — user422005, Sep 25 '10 at 15:42
I had a problem importing stdin/stdout/stderr from an SO, and there was no chance I was going to build-out a formal Structure. Your fdopen() suggestion saved me. Thanks. — Dustin Oprea, Sep 19 '13 at 02:49

score 3 · Answer 2 · answered Mar 02 '16 at 10:18

If you want to use stdout/stdin/stderr, you can import those variables from the standard C library.

libc = ctypes.cdll.LoadLibrary('libc.so.6')
cstdout = ctypes.c_void_p.in_dll(libc, 'stdout')

Or, if you want to avoid using void* for some reason:

class FILE(ctypes.Structure):
    pass

FILE_p = ctypes.POINTER(FILE)

libc = ctypes.cdll.LoadLibrary('libc.so.6')
cstdout = FILE_p.in_dll(libc, 'stdout')

score 2 · Answer 3 · edited May 21 '15 at 00:55

Well;

I tried the fileno based solution, but was quite uncomfortable with opening the file twice; It was also not clear to me how to avoid the return value from fdopen() to leak.

In the end I wrote a microscopic C-extension:

static PyObject cfile(PyObject * self, PyObject * args) {
    PyObject * pyfile;
    if (PyArg_ParseTuple( 'O' , &pyfile)) {
       FILE * cfile = PyFile_AsFile( pyfile );
       return Py_BuildValue( "l" , cfile );
    else
        return Py_BuildValue( "" );
}

which uses PyFile_AsFile and subsequently returns the FILE * pointer as a pure pointer value to Python which passes this back to C function expecting FILE * input. It works at least.

Joakim

score 0 · Answer 4 · answered May 20 '16 at 22:25

0

Adapted from svplayer

import sys

from ctypes import POINTER, Structure, py_object, pythonapi


class File(Structure):
    pass

if sys.version_info[0] > 2:
    convert_file = pythonapi.PyObject_AsFileDescriptor
    convert_file.restype = c_int
else:
    convert_file = pythonapi.PyFile_AsFile
    convert_file.restype = POINTER(File)

convert_file.argtypes = [py_object]

fp = open('path').fp
c_file = convert_file(fp)

answered May 20 '16 at 22:25

reubano

5,087
1
42
41

Except this wrong: for py3k ``c_file`` will be ``int``, for py2k it will be ``FILE *``. – mcepl Nov 14 '16 at 22:02
@mcepl you mind editing the answer to make it correct? – reubano Nov 15 '16 at 13:20
I believe the correct answer is in my solution above. – mcepl Nov 15 '16 at 19:00

score 0 · Answer 5 · answered Nov 14 '16 at 22:20

Tried this:

#if PY_MAJOR_VERSION >= 3
    if (PyObject_HasAttrString(pyfile, "fileno")) {
        int fd = (int)PyLong_AsLong(PyObject_CallMethod(pyfile, "fileno", NULL));
        if (PyObject_HasAttrString(pyfile, "mode")) {
            char *mode = PyUnicode_AsUTF8AndSize(
                    PyObject_CallMethod(pyfile, "mode", NULL), NULL);
            fp = fdopen(fd, mode);
        }
        else {
            return PyErr_Format(PyExc_ValueError,
                    "File doesn’t have mode attribute!");
        }
    }
    else {
        return PyErr_Format(PyExc_ValueError,
                "File doesn’t have fileno method!");
    }
#else
    fp = PyFile_AsFile(pyfile);
#endif

It looks like it might be working.

score -1 · Accepted Answer · answered Apr 14 '11 at 14:56

-1

I've encountered the same problem.

Take a look at this file:

http://svn.python.org/projects/ctypes/trunk/ctypeslib/ctypeslib/contrib/pythonhdr.py

You can use PyFile_AsFile from it.

answered Apr 14 '11 at 14:56

yk4ever

791
5
7

Thanks - that was a better solution than mine. – user422005 Apr 15 '11 at 18:37
5

Unfortunately PyFile_AsFile doesn't exist in Python 3 C API. – Mikhail Korobov Jul 16 '12 at 10:27
2

@MikhailKorobov: I asked [a question](http://stackoverflow.com/q/16130268/309483) to resolve this issue. – Janus Troelsen Apr 21 '13 at 10:37

Python ctypes: Python file object <-> C FILE *

6 Answers6

Linked