1

On local development app-engine server(and on production app engine), I want to run a script which has big-query related code, but I am getting an error. I have already visited all the answers related to this error in stackoverflow but did not find anything workable. following is the code and error that occurred. Thanks in advance for your precious time.

Code

#!/usr/bin/python 
from google.cloud import bigquery
client = bigquery.Client('myproject')
query = '(SELECT *FROM (SELECT *, ROW_NUMBER() OVER (PARTITION BY tb1.user_id ORDER BY tb1.timestamp DESC) AS RowNo FROM `mytable` tb1) x WHERE x.RowNo = 1)'
job_config = bigquery.QueryJobConfig()
job_config.allow_large_results = True
job_config.destination = dest_table_ref
job_config.write_disposition = 'WRITE_TRUNCATE'
query_job = client.query(query, job_config=job_config)

When I hit the url (http://myproject:8080/cronscript) I get following errors

Error

File "/home/username/src/project-name/python_reporting-2017-12-15-10-37/cronscript.py", line 3, in from google.cloud import bigquery File "/google/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/runtime/sandbox.py", line 1132, in load_module raise ImportError('No module named %s' % fullname) ImportError: No module named google.cloud.bigquery

However, when I open python shell by typing python and then if I type

from google.cloud import bigquery

it works fine. If I run python cronscript.py it also gets executed properly $ python cronscript.py Done $

Only when I execute this from a local app server after performing dev_appserver.py app.yaml I receive above error.

What all I have already tried

1) sudo pip install --upgrade google-cloud-bigquery
2) my requirements.txt look like
    google-api-python-client
    google-cloud
3) pip install -t lib/ requirements.txt
4) from google.appengine.ext import vendor
   vendor.add('lib')
5) from __future__ import absolute_import

Still unresolved.

Virendra Goswami
  • 113
  • 1
  • 2
  • 6
  • Is the error prompted when you run it with the Local Development Server or on production (App Engine)? Are you using AppEngine Flex or Standard? Which Python version re you running? – dsesto Dec 22 '17 at 09:28
  • On both the server, cloud shell testing server and production app engine I got this error – Virendra Goswami Dec 22 '17 at 10:25
  • I'm not sure if this is still the case but at one point the Google python client libraries had naming conflicts (super frustrating!). See this question: https://stackoverflow.com/questions/41399303/using-two-python-libraries-with-conflicting-names – afed Dec 22 '17 at 16:28
  • It successfully gets executed when I type command e.g. python cronscript.py . But via server call(i.e. http://myproject:8080/cronscript) it throws above error – Virendra Goswami Dec 23 '17 at 05:07

1 Answers1

2

It looks like you don't have this module. So, did you try to install/upgrade your google cloud?

sudo pip install --upgrade google-cloud-bigquery
Victor M Perez
  • 2,185
  • 3
  • 19
  • 22
Vasily Bronsky
  • 435
  • 2
  • 12