Google appengine project Third party library import error

Question

I am developing an app-engine project in Google App Engine Standard Environment Python 2.7. Mainly my application using webapp2 for request handling(WSGI protocol) and jinja2 as the python frame work.

I am getting no module name error when try to import google-cloud-bigquery, google-auth, google-oauth2.

These are the solutions i have tried so far to fix but could not success anyone of them.

Here's the previous [question]: Google Cloud BigQuery Import not working in app engine project

Second one [question]: Error importing Google Cloud Bigquery api module in python app

Google document [a link]: https://cloud.google.com/appengine/docs/standard/python/tools/using-libraries-python-27

app.yaml

runtime: python27
threadsafe: true

libraries:
- name: webapp2
 version: latest
- name: jinja2
  version: latest
- name: PIL
  version: latest

handlers:
- url: /fav_icon
  static_dir: fav_icon

- url: /fav_icons
  static_dir: fav_icons

- url: /css
  static_dir: css
  mime_type: 'text/css'

 - url: /js
   static_dir: js

- url: /.*
  script: main.app

main.py

from __future__ import absolute_import
import sys
sys.path.insert(0, 'lib')
import os
import jinja2
import webapp2
import csv
from jinja2 import Template
import flask
from google.cloud import bigquery
import google.auth
from google.oauth2 import service_account
JINJA_ENVIRONMENT = jinja2.Environment(
# TODO: to add other directories here that contains the templates.
loader=jinja2.FileSystemLoader(os.path.join(os.path.dirname(__file__),'templates')),
extensions=['jinja2.ext.autoescape'],
autoescape=True)

class Bigquery(webapp2.RequestHandler):
     """docstring for ."""
     def get(self):
         ser_acc_file_path = os.getcwd()+ str('/file_name.json')
         credentials = service_account.Credentials.from_service_account_file(ser_acc_file_path)
         if credentials.requires_scopes:
           credentials =credentials.with_scopes(['https://www.googleapis.com/auth/bigquery'])
          bigquery_client = bigquery.Client(project='project_123', credentials=credentials)
          query_results = bigquery_client.run_sync_query("""SELECT * FROM `project_123.dataset.table` LIMIT 100;""")
          query_results.use_legacy_sql = False
          query_results.run()
          rows = query_results.fetch_data()
          self.response.headers['Content-Type'] = 'text/plain'
          self.response.write(str(row))
app = webapp2.WSGIApplication(routes=[
('/',Bigquery))],debug=True)

if __name__ == '__main__':
main()

All the third party libraries are installed in lib directory using pip.

Answer 1

See my answer to a similar question here. Here is a link to the docs for using third-party libs in GAE. Basically, the solution is you make a new file called appengine_config.py in the same directory as your app.yaml, and that file should look like this:

# appengine_config.py
import os
from google.appengine.ext import vendor

# Add any libraries installed in the "lib" folder.
vendor.add(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib'))

Also, I don't know if this is causing problems at the moment, but you should remove the the last two lines of your main.py, the ones that look like if __name__ == __main__. When you run your app either with dev_appserver.py or by deploying it to GAE, it will establish the wsgi app for you, set up the handlers according to app.yaml and your Routes, etc. It should not be using app.py as your main module.