1

I am trying to build a web app on Google App Engine (GAE) using python and Flask. I want to know how to call a function within the app multiple times simultaneously. For example, I have a function which returns the square of an element. Now, I wrap this function under a URL like this:

@app.route('/single_number',methods = ['GET','POST'])
def square():
    x = request.form.get('input_number')
    return x*x

Now, I want to do this on a list of numbers but concurrently on 'http://localhost:8080/list_of_numbers'. Say, I have a list of 'n' numbers and I want to calculate the square of each number simultaneously (if not all 'n' together but at least what Google App Engine allows). I have used multiprocessing library of Python but got no performance improvement because of the single core used in Google App Engine instance and apparently, multithreading is equally futile. So, upon reading some more I was led to Cloud Tasks of GAE. But I am not able to wrap my head around executing this on Cloud Tasks. Any sample/tutorial related to this will be greatly appreciated. If you think that Cloud Tasks is not the right way for this problem, is it possible to create a function under handler '/list_of_numbers', which will send multiple numbers of the list to the '/single_number' handler simultaneously? Or do you suggest any other method?

I am new to web-app building and GAE. So, any help would be greatly appreciated.

Thanks in advance

Amruth Kiran
  • 31
  • 1
  • 8

1 Answers1

1

This is what it would look like using google taskqueue (which is v1 of Google Cloud Tasks)

    from google.appengine.api import taskqueue

    queue = taskqueue.Queue('default')
    ndb.Future.wait_all(queue.add_async(taskqueue.Task(
        url='/single_number/,
        params={'input_number': i})) for i in input_numbers)

Which will add a batch of tasks to the queue. However this won't receive the results of the calls to /single_number/ so each task will have to handle whatever comes next.

The answer to your question really depends on what you are trying to do and the expected scale of your problem.

It feels like you want to use something like Google Dataflow

Alex
  • 5,141
  • 12
  • 26
  • Thanks. I will try this method. But I always get 'ModuleNotFoundError: No module named 'google.appengine' ' when I try to test it locally (in a virtual environment) and even when I deploy the app to test it. How do I overcome this error for deploying? – Amruth Kiran Jan 31 '19 at 11:14
  • Are you on app engine standard? is this your first time importing anything from `google.appengine` (i.e. you dont use `ndb` or `deferred` or `blobstore` etc)? – Alex Jan 31 '19 at 16:36
  • Yes, I am on standard. Yes, this will be my first time importing something from google.appengine. – Amruth Kiran Jan 31 '19 at 17:37
  • which version of python – Alex Jan 31 '19 at 18:00
  • I believe these libs are only available for python 2.7. There are a few reasons why your environment isn't happy see this question https://stackoverflow.com/questions/15783783/python-import-error-no-module-named-appengine-ext or this one https://stackoverflow.com/questions/36022960/google-app-engine-importerror-no-module-named-appengine-ext. Maybe start with one of the sample projects Google cloud provides that uses these libraries and try to get that working. – Alex Jan 31 '19 at 18:06