Adding a third-party library (twilio) to project using Google App Engine and Django

Question

Everyone.

I'm a newbie in this field. I develops web application with google app engine using django framework. I have a troubleshot about python lib dir problem... ImportError: no module named...

my appengine_config.py file is

# [START vendor]
from google.appengine.ext import vendor 

vendor.add('lib') # I believes this line is to add 'lib' folder to PATH.

# vendor.add(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib')) # <-- and I tried too this line.

# [END vendor]

my 'requirements.txt' file is

MySQL-python==1.2.5 #app engine django project default
Django==1.11.3 #app engine django project default
django-twilio # add i want
twilio # add i want

and I installed using pip install -t lib -r requirements.txt

ROOT
├── lib
│   ├── django
│   ├── pytz
│   ├── wanttousing_lib
│   └── ...
├── mysite
│   ├── __init__.py
│   ├── settings.py
│   ├── controllers.py
│   ├── models.py
│   ├── views.py
│   ├── templates
│   └── ....
├── test
│   ├── like
│   │   ├── models_tests.py
│   │   └── controllers_tests.py
│   └── ....
├── static
│   ├── css
│   └── js
├── app.yaml
├── manage.py
├── appengine_config.py
├── requirement-vendor.txt
└── requirements.txt

so, I installed in my project... but..compiled error.

from wanttousing_lib import example_module importError wanttousing_lib..........

however, if I move my wanttousing_lib to ROOT dir, it works.....

ROOT
├── lib
│   ├── django
│   ├── pytz
│   
│   └── ...
├── mysite
│   ├── __init__.py
│   ├── settings.py
│   ├── controllers.py
│   ├── models.py
│   ├── views.py
│   ├── templates
│   │   └── like
│   │        ├── index.html
│   │        └── _likehelpers.html
│   └── ....
├── test
│   ├── like
│   │   ├── models_tests.py
│   │   └── controllers_tests.py
│   └── ....
├── static
│   ├── css
│   └── js
├── app.yaml
├── manage.py
├── appengine_config.py
├── requirement-vendor.txt
├── requirements.txt
└── wanttousing_lib  <--- moved

--> All traceback.

Unhandled exception in thread started by <function wrapper at 0x103e0eaa0>
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/utils/autoreload.py", line 227, in wrapper
    fn(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/commands/runserver.py", line 125, in inner_run
    self.check(display_num_errors=True)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/base.py", line 359, in check
    include_deployment_checks=include_deployment_checks,
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/base.py", line 346, in _run_checks
    return checks.run_checks(**kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/checks/registry.py", line 81, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/checks/urls.py", line 16, in check_url_config
    return check_resolver(resolver)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/checks/urls.py", line 26, in check_resolver
    return check_method()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/urls/resolvers.py", line 254, in check
    for pattern in self.url_patterns:
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/utils/functional.py", line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/urls/resolvers.py", line 405, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/utils/functional.py", line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/urls/resolvers.py", line 398, in urlconf_module
    return import_module(self.urlconf_name)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "ROOT/mysite/urls.py", line 19, in <module>
    from polls.views import index
  File "ROOT/polls/views.py", line 17, in <module>
    from sms_twilio.tests import send_sms_test
  File "ROOT/sms_twilio/tests.py", line 13, in <module>
    from twilio import twiml
ImportError: No module named twilio

ERROR SOURCE:

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.test import TestCase

# Create your tests here.
from django.conf import settings

# file: your_code.py
# import twilio  # no need for 'from lib import twilio'
# do stuff with twilio...

from twilio import twiml
from twilio.rest import Client


def send_twilio_message(to_number, body):
    client = Client(
    #client = twilio.rest.TwilioRestClient(
        settings.TWILIO_ACCOUNT_SID, settings.TWILIO_AUTH_TOKEN)

    return client.messages.create(
        body=body,
        to=to_number,
        from_=settings.TWILIO_PHONE_NUMBER
    )

def send_sms_test():
    client = Client(
    #client = twilio.rest.TwilioRestClient(
        settings.TWILIO_ACCOUNT_SID, settings.TWILIO_AUTH_TOKEN)

    return client.messages.create(
        body="[TEST] SEND SMS !! HELLO !!",
        to="TO_SENDER",
        from_=settings.TWILIO_PHONE_NUMBER
    )

perhaps, Do I add library list to app.yaml ? like

libraries:
- name: MySQLdb
  version: 1.2.5
- name: twilio <-- like this
  version: -

requirement-vendor.txt file is

Django==1.11.3

how can i fix it? please help...

Answer 1

I had a similar issue a while back, and instead of using vendor.add('lib'), I had success doing this:

vendor.add(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib'))

I found that solution here in the docs for using third-party libraries on GAE, in step 4:

To copy a library into your project:

1. Create a directory to store your third-party libraries, such as lib/.

   mkdir lib
   

2. Use pip (version 6 or later) with the -t flag to copy the libraries into the folder you created in the previous step. For example:

   pip install -t lib/ <library_name>
   

   (Using Homebrew Python on Mac OS X?)

3. Create a file named appengine_config.py in the same folder as your app.yaml file.

4. Edit the appengine_config.py file and provide your library directory to the vendor.add() method.

   # appengine_config.py
   from google.appengine.ext import vendor
   
   # Add any libraries install in the "lib" folder.
   vendor.add('lib')
   

   The appengine_config.py file above assumes that the current working directory is where the lib folder is located. In some cases, such as unit tests, the current working directory can be different. To avoid errors, you can explicity pass in the full path to the lib folder using:

   vendor.add(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib'))
   

Answer 2

My python lib's dir are two dir.

1) /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/...

2) /usr/local/lib/python2.7/...

My project points at 1), but pip install at 2)...

I tried at 1) ./pip install twilio. and so, it works!

thanks.