0

I have a Python Google App Engine project structured as follows:

app/
    handlers/
        register_user.py
    models/
        user.py

The user.py file contains a class User(ndb.Model).

I'm trying to access the User class from register_user.py to create and put a new user in the database. Normally, I'd just import it like this:

from ..models.user import User

But this errors out because I'm trying to import something from above my root package - so I'm guessing models is my root package, and I can't get back to the app package?

Right now, I'm able to work around it by importing like this:

import importlib
User = importlib.import_module('models.user').User

I think this is kind of messy, though. So what's the "right" way of importing my User class?

Edit: The full stack trace:

Attempted relative import beyond toplevel package (/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py:1552)
Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~polly-chat/1.394430414829237783/main.py", line 48, in post
    receive_message(messaging_event)
  File "/base/data/home/apps/s~polly-chat/1.394430414829237783/messaging/handler.py", line 39, in receive_message
    intent_picker.respond_to_postback(messaging_event)
  File "/base/data/home/apps/s~polly-chat/1.394430414829237783/messaging/intent_picker.py", line 71, in respond_to_postback
    intent = importlib.import_module('intents.register_user')
  File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/base/data/home/apps/s~polly-chat/1.394430414829237783/intents/register_user.py", line 1, in <module>
    from ..models import messenger_user
ValueError: Attempted relative import beyond toplevel package

(The package names here are different; I simplified them above to make the example more general)

Dan Cornilescu
  • 39,470
  • 12
  • 57
  • 97
Leon Overweel
  • 1,488
  • 4
  • 17
  • 27

2 Answers2

1

I think Dan is on the right track, however there is no need to vendor your own code. The vendoring system is to manage third-party dependencies with pip and should be totally unnecessary for your use case, and vendoring your own code would violate the conventions.

Based on what you've told us, you should be able to import your code with just

from models import user

If that doesn't work, you should figure out why, but you definitely do not need the vendor extension or importlib to solve it.

Your base module is wherever your base WSGI application is located, which is going to be defined by where your app.yaml routes to. Typically, your app.yaml will contain something like:

- url: .*  # This regex directs all routes to main.app
  script: main.app

In this case, in the same directory as app.yaml there is a main.py that contains an app WSGI application. In some other cases, script might be application.main.app, in which case the app variable is in application/main.py and then the application directory would be the base directory.

Every Python package containing modules should contain an __init__.py file in its directory, as Dan mentioned. As a side note if you do use a lib directory for third-party code it won't contain an __init__.py since it's not a Python package (just a directory that contains Python packages). The fact that its not a package is why you use the vendor extension Dan described to make sure the packages it contains are on the import path.

In my experience, relative imports are rarely needed and can get you into problems like these so I would just avoid them.

If you're still stuck, lay out the entire file structure of your application, including the app.yaml contents, and each of the subdirectories, including whether they contain an __init__.py or not.

Bill Prin
  • 2,498
  • 1
  • 20
  • 27
  • I think I was doing my non-relative imports wrong before, but now they work! `from models.user import User` correctly imports from `register_user.py`. – Leon Overweel Jul 24 '16 at 23:15
0

The way I approached this was using the GAE 3rd party lib vendoring technique:

  • created appengine_config.py:

content:

from google.appengine.ext import vendor

# Add any libraries installed in the "lib" folder.
vendor.add('lib')
  • created the /app/lib dir
  • added an empty __init__.py file in the models dir to make it a package
  • placed/moved/symlinked the models dir inside the /app/lib dir

With this the models can be referenced using:

from models.user import User

Possibly of interest:

Community
  • 1
  • 1
Dan Cornilescu
  • 39,470
  • 12
  • 57
  • 97
  • Vendoring is for third-party code, there should be no need to use the vendor extension for your own code (and in fact doing so violates the conventions of it in the first place). – Bill Prin Jul 24 '16 at 22:37
  • @BillPrin I admit, I found it convenient and just went for it. What do you mean by "violates the conventions" - violation of some Google policy or just violation of some convention? – Dan Cornilescu Jul 25 '16 at 01:11