Import in python 3 complains about argument as a str / bytes

Question

I've being updating a quaternions package for integration with numpy, so that it can be used in both python 2 and python 3. Unfortunately, the basic import step fails miserably with 3.x, though it has never failed with python 2.7. (I use python2.7 to compile the 2.7 version, and python3.x to compile the 3.x versions. It's a really simple distutils thing.) The error message doesn't even appear in google's results, and I just have no idea where to go from here.

Here is the complete output from a simple attempt to import the package:

> python -c 'import quaternion'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/mynamehere/.continuum/anaconda/envs/py3k/lib/python3.4/site-packages/quaternion/__init__.py", line 3, in <module>
    from .numpy_quaternion import quaternion
TypeError: __import__() argument 1 must be str, not bytes

As the error message says, there is a line in __init__.py saying

from .numpy_quaternion import quaternion

But why should that be problematic? There is a file numpy_quaternion.so in the same directory as the __init__.py file, which seems to contain the relevant symbols. Travis-CI shows that it works just fine in 2.7 (and the other tests pass), but fails in 3.2 and 3.4. So it's not just something wrong with my python installation. I tried to remove the . for the relative import, but python couldn't find the numpy_quaternion from which to import (not surprising). I tried changing it to from quaternion.numpy_quaternion, but I get the same error.

I see that there have been changes to the import system in python 3, but if anything, I would have guessed that this would be more py3k-compliant than other ways of doing it. What's going wrong? How can I get this to work?

Just to clarify, my hierarchy looks like this:

.../site-packages/
    quaternion/
        __init__.py
        numpy_quaternion.so

and the only thing that comes before the problematic line is import numpy as np, which generally succeeds with no problem.

Mike · Accepted Answer · 2014-10-22T18:51:28.433

4

The python-list people got back to me right away with excellent suggestions. Turns out I was importing something within numpy_quaternion.so (using the c-api), but the argument I was giving to that function was wrong. I was (basically) using code from a similar package:

PyObject* numpy_str = PyString_FromString("numpy");
PyObject* numpy = PyImport_Import(numpy_str);

I fixed it by using

PyObject* numpy = PyImport_ImportModule("numpy");

And as J. F. Sebastian points out in the comments, the reason that was going wrong for me was because that PyString_FromString was just a #define for the wrong function when I was using python 3.

edited Oct 22 '14 at 18:51

answered Oct 21 '14 at 00:54

Mike

19,114
12
59
91

1

Note: [Porting Extension Modules to Python 3](https://docs.python.org/3/howto/cporting.html) : *"Python 3's `PyString_*` functions in C are equivalent to Python 2's `PyUnicode_*`"*. Use `PyString` for Unicode text, `PyBytes` for binary data. The TypeError in your question suggests that you use PyBytes on Python 3 somehow instead of PyString. – jfs Oct 21 '14 at 02:41
Aha! Not directly, but I *do* include `npy_3kcompat.h`, which does `#define PyString_FromString PyBytes_FromString`. So it looked right to me, but it wasn't; in fact `PyString_FromString` isn't even defined in python3. – Mike Oct 21 '14 at 03:28

score 0 · Answer 2 · answered Oct 20 '14 at 21:59

Since it is easy, I would first try an absolute import, though if my guesses below are correct, this will not work.

from quaternion.numpy_quaternion import quaternion

From your post, I am guessing that your hierarchy looks like

.../Libe/site-packages
    quaternion
    __init__.py
    numpy_quaternion.so
        quaternion  # a symbol in .so, not a .py

and that quaterion is a module, rather than a function or class. I am guessing this because I cannot imagine 'numpy_quaternion' becoming bytes, while the .so must return 'quaternion' as bytes for 2.7 to work, so maybe it is doing the same with 3.x. My unix experience predates Python. But my impression is that separate .so are needed for 2.x and 3.x. Or if not, certain compile flags might be needed. If I am correct, you need to add 'numpy_quaternion_3x.so to your package and switch the import on sys.version[0].

If you do not get more response here, try python-list, easily accessed at news.gmane.com as newsgroup mirror gmane.comp.python.general. The regular responders include some savvy linux users.

Yeah, I've tried the absolute import. (Mentioned in my post, but I didn't make it clear.) Also, there are separate `lib` directories for py2.7 and py3.x, and each is used to compile for its own directory, so they do get their own `.so` files. (py3.x doesn't even know py2.7 exists.) I'll try on the python-list, though. Thanks! — Mike, Oct 20 '14 at 22:22

Import in python 3 complains about argument as a str / bytes

2 Answers2