Dowser: "bad argument to internal function"

Question

I'm trying out Dowser, it's really cool but I'm stuck with a little problem, and I couldn't find anything helpful on Google, so here I am.. ^^;

I'm running a CherryPy+SQLAlchemy application.. It works normally, except that when I enable Dowser (that is, after I call dowser.Root()), now and then I get exceptions like:

SystemError: ../Objects/tupleobject.c:809: bad argument to internal function

on innocent-looking istructions, such as accessing a SQLA-mapped field.. Relevant part of a traceback:

  File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/attributes.py", line 158, in __get__
    return self.impl.get(instance_state(instance), instance_dict(instance))
  File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/attributes.py", line 377, in get
    value = callable_()
  File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/strategies.py", line 586, in __call__
    result = q.all()
  File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/query.py", line 1267, in all
    return list(self)
  File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/query.py", line 1422, in instances
    rows = [process[0](context, row) for row in fetch]
  File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/query.py", line 2032, in main
    return _instance(row, None)
  File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/mapper.py", line 1653, in _instance
    identitykey = identity_key(row)
  File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/mapper.py", line 1594, in identity_key
    return (identity_class, tuple(row[column] for column in pk_cols))

Could this be related to the Dowser thread, accessing the garbage collector? Any hint on what I could check out?

I'm running Python 2.6.2 on Xubuntu Jaunty.

Thanks for your attention!

13 years later... and it's still there! An application on Python 3.8.10, SQLAlchemy 1.4.25, and Dozer 0.8 running for a few hours crashes with the same `SystemError: ../Objects/tupleobject.c:892: bad argument to internal function`. — saaj, Jun 05 '23 at 16:08

saaj · Answer 1 · 2023-06-05T19:23:06.760

Abstract

In my case it's Dozer, not Dowser, but it's basically the same thing -- looping over gc.get_objects and counting types in a separate thread. And optionally tracing referrers and referents of the chosen objects interactively.

As I mentioned in the comment this error still takes place. On SQLAlchemy I had two stack traces of SystemError: ../Objects/tupleobject.c:892: bad argument to internal function exception raised in tuple comprehensions (one, two). Here's a plausible explanation.

Explanation

If I look at CPython source of tupleobject.c line 892, there's a notable comment preceding the function:

The following function breaks the notion that tuples are immutable: it changes the size of a tuple. We get away with this only if there is only one module referencing the object. You can also think of it as creating a new tuple object and destroying the old one, only more efficiently. In any case, don't use this if the tuple may already be known to some other part of the code.

Proposition:

while SQLAlchemy is building its complex tuple comprehension in the main thread,
Dozer grabs a reference to the "work-in-progress" tuple, from another thread,
and breaks the assumption of _PyTuple_Resize (that there's only 1 referrer for non-empty tuple)

    if (v == NULL || Py_TYPE(v) != &PyTuple_Type ||
        (Py_SIZE(v) != 0 && Py_REFCNT(v) != 1)) {
        *pv = 0;
        Py_XDECREF(v);
        PyErr_BadInternalCall();  // line 892
        return -1;
    }

Takeaway

Dozer is a great live memory inspection tool, but it's limited in the way that the application under inspection instrumented with Dozer may not survive a long run (e.g. survives a few hours). Hence, it's more practical to wait until the memory leak manifests and only then live attach Dozer to the process.

Dowser: "bad argument to internal function"

1 Answers1

Abstract

Explanation

Takeaway