3

I am using Google Cloud Functions to run a Python function which is being hit by my Facebook Messenger bot. Facebook Messenger has a weird requirement:

When you receive a webhook event, you must always return a 200 OK HTTP response. The Messenger Platform will resend the webhook event every 20 seconds, until a 200 OK response is received. Failing to return a 200 OK may cause your webhook to be unsubscribed by the Messenger Platform.

In other words, I have 20 seconds to return a 200 OK response from my cloud function. The issue is I need to actually send the data I receive from the message to another API using a requests.post, and there is basically no guarantees that it will take less than 20 seconds. So ideally, I would want to use threads and multi-processing. I realize that Google does not allow for threading. What can be a way to handle this situation? Thanks.

darkhorse
  • 8,192
  • 21
  • 72
  • 148
  • 4
    This question is a good programming question. You must understand your design requirements to write code. Why the downvotes without providing feedback? If this question is to be closed, provide some input so that darkhorse can write an improved question. I want to see the best answer for this question. – John Hanley May 12 '19 at 22:00

1 Answers1

7

If I am understanding correctly, we have an original REST caller (Facebook) that requires a response in 20 seconds or less. However, your function that is handling the request wishes to call a service that may take longer than 20 seconds to respond. One possible solution is to realize that the the response sent by the original Cloud Function can be returned even before the code logic in that Cloud Function completes. Assuming you are coding in Node.js, you are passed a response object to the Cloud Function. You can call send() and end() on this object within the Cloud Function body and then carry on and do your other long-running work. If you are wanting to return a success criteria on the overall process to your original caller, it sounds like we have conflicting stories. The contract seems to say that the response must be 20 seconds or less while the work may take more than 20 seconds. As such, you can't sensibly return a response indicating full outcome in all cases.

If you are writing your application in Python, the story is not so easy. Cloud Functions leveraging Python is based of flask and with flask, a Python function is called (that you supply). The data returned from this function is the data that is returned to the caller. If we want to return early and THEN do additional work, one recipe is described here. Unfortunately, it doesn't look like we can avail ourselves of this story because Cloud Functions owns and manages the Flask environment and we don't have a legal way to add additional handlers.

Another idea that occurs to me is to have your Python code that is invoked by Facebook that now runs in Cloud Functions execute a Google Pub/Sub publish request passing the data that you need to pass. This publish will return immediately and you can then quickly return from your Cloud Function and satisfy the original request. The published message could then be used to trigger a second cloud function that could take as long as it needs to complete.

One further thought is to not use Cloud Functions at all. The chances are high that you are using Cloud Functions to achieve serverless coding and to not have to worry about scaling considerations. A new feature (as of Next 19) is now available called Cloud Run that allows you to create a Docker container that could run your own Flask python app and then you would have full control over the environment. It seems that what we have is a trade-off. We have Cloud Functions that provides a super-quick mechanism for handling the vast majority of REST requests ... however, if we need specialization (MUST respond in 20 seconds or less and do follow-on work), then Cloud Function may have tipped into a hindrance should we want to achieve some custom processing.

Kolban
  • 13,794
  • 3
  • 38
  • 60
  • Thats interesting. And yes, I can't sensibly return a response indicating full outcome. However, I'm coding in Python, so what would be Python way of doing send() and end()? – darkhorse May 12 '19 at 20:12
  • Added thoughts on Python and other alternatives. – Kolban May 12 '19 at 20:31
  • Your advice is very good. My input: Cloud Functions is the wrong approach - the HTTP return ends function execution. Cloud Run is also the wrong approach as the CPU is idled to zero in between HTTP requests. This is a task for App Engine Standard or Firebase if the goal is serverless. App Engine Flex is another good non-serverless approach. – John Hanley May 12 '19 at 21:57