Check if object is iterable in python c extension

Question

I need to write a function that does different things depending on whether a python argument is iterable or not. Here's a code example:

PyObject *iter = PyObject_GetIter(arg); // arg is a PyObject*
if (iter) {
  // do iterable things
} else {
  // do non-iterable things
}

It appears, though, that if arg is not iterable, then not only is iter == NULL, but an exception is also thrown. What is the right way to handle this situation? Do I just call PyErr_Clear() and hope that no other errors have been set?

As far as I understand, `PyIter_Check()` checks if `next` can be called on an object. So, it's not gonna work here. In my example, `PyIter_Check()` would return the correct results when called on `iter`, not on `arg`. In fact, I've tried this, and it always returns false for `arg`, whether it's a number or a list. I need to test whether `arg` is a list, or a tuple, or any other container. There's no next for a tuple. — SU3, Mar 01 '20 at 04:30
It may just be a call to `PyErr_Clear()` to get rid of the exception. @falsetru - does that also check if its iterable or just whether its already an iterator? — tdelaney, Mar 01 '20 at 04:31
Sorry, I mis-read the question. There was a code calls `PyObject_GetIter`, and `PyErr_Clear` in [`Python/import.c`](https://github.com/python/cpython/commit/c1a6832f50b36ffec299e6e6038535904e2b158d#diff-12fb18f1056d99e8480487a481f55362L483-L485). It's now replaced with `PyErr_WriteUnraisable()`. I think it's safe to call `PyErr_Clear` in your case. — falsetru, Mar 01 '20 at 09:00

score 2 · Answer 1 · edited Sep 15 '20 at 10:30

In pure Python, the usual solution for finding out whether an object is iterable is to call iter(...) and to see what happens (it was for example popularized by "Fluent Python"):

def is_iterable(obj):
    try:
        iter(obj)  # ok, it worked
        return True
    except TypeError:
        return False

for more details, see this great answer.

This is also basically the @falsetru's proposal in comments - to try and to clear error, if PyObject_GetIter fails:

int is_iterator(PyObject *obj){
    PyObject *it =  PyObject_GetIter(obj);
    if(it != NULL){
        Py_DECREF(it);
        return 1; // object can be iterated
    }
    else if (PyErr_ExceptionMatches(PyExc_TypeError)) {
        PyErr_Clear();
        return 0; // is not an iterator
    }
    else{
        return -1; // error
    }
}

But it might not be what you mean by "being iterable", then one can adjust the implementation of PyObject_GetIter to your needs, for example:

int is_iterator2(PyObject *obj) {
    return Py_TYPE(obj)->tp_iter != NULL || PySequence_Check(obj);
}

As in usual algorithm, is_iterator2 looks up if the tp_iter-slot (i.e. __iter__-function) is present and if not falls back to the sequence protocol via __getitem__. However, differently as in the first version, the tp_iter slot isn't called and its result isn't checked for being an iterator, i.e. for

class C:
    def __iter__(self):
        raise BufferError()
        
class D:
    def __iter__(self):
        return 1; # isn't iterator

C() and D() would be classified as iterable (which would not be the case for the first is_iterator version). Also if is_iterator2 returned 1, it doesn't mean PyObject_GetIter doesn't return NULL as can be seen for the above classes.

Check if object is iterable in python c extension

1 Answers1