2

I have been using Google Cloud Video Intelligence annotation functionality with Google App Engine Flex. When I try to use VideoIntelligence with a two hour video, it takes 60 minutes for the AnnotateVideo function to respond.

gs_video_path ='gs://'+bucket_name+'/'+videodata.video.path+videodata.video.name
print(gs_video_path)

video_client = videointelligence.VideoIntelligenceServiceClient()
features = [videointelligence.enums.Feature.OBJECT_TRACKING]
operation = video_client.annotate_video(gs_video_path, features=features)

Currently, the only place I can execute this is on Google App Engine Flex. Yet, Google App Engine Flex keeps an instance idle all the time, it is very similar to running a VM in terms of cost.

Google App Engine has a 540 seconds timeout and same wise Google Cloud Run has a timeout of 900 seconds and Google Cloud Functions has a max timeout of 600 seconds as far as I understand.

In these circumstances, which Google Cloud product should I use for a one hour process that should take place while avoiding having an idle instance when there is no usage.

(Please don't respond quoting GKE or other VM based solutions, no idle instance solutions are accepted)

london_utku
  • 1,070
  • 2
  • 16
  • 36

3 Answers3

2

Cloud Run’s 900 second timeout is subject to change soon to satisfy your needs (up to an hour). There's a feature in the works. I’ll update here once it's available in beta, stay tuned.

#ahmetb-todo

ahmet alp balkan
  • 42,679
  • 38
  • 138
  • 214
  • 1
    You have the same comment here in 2019, 16th May. https://stackoverflow.com/questions/56139697/how-to-increase-15-minutes-request-timeout-for-google-cloud-run-service-deployed – london_utku May 06 '20 at 16:22
  • And also there are cases where one hour is not sufficient as well, there should be a real pay as you use "google cloud function like" service which should fulfil all transcode and video processing requests that can require long durations of execution. – london_utku May 06 '20 at 16:26
  • 1
    I actually don't have the same comment there, as that question is about Cloud Run on GKE. You asked about 60 minutes in your question, that’s why I’ve brought this up. – ahmet alp balkan May 06 '20 at 20:04
  • UPDATE: According to https://cloud.google.com/run/docs/configuring/request-timeout Cloud Run already accepts 60 minutes. – Alexander Santos Mar 11 '22 at 12:41
1

I don't think Google has a service that matches your needs.
Probably you should implement some custom workflow, like:

  1. From "short living" environment, like Function, CloudRun or AppEngine do following:
    1. Put an event for your long-running task to PubSub
    2. Use Compute Engine API to start a VM
  2. When VM starts, it's startup script should get latest item from the PubSub and start your long-running task
  3. When task is done, VM terminates itself using ComputeEngine API or calls a Function that invokes shutdown
Slava Medvediev
  • 1,431
  • 19
  • 35
1

You can specify output_uri in the original request. This will write the final result to your GCS bucket. Then you don't have to wait for the long running operation to finish on your VM. The initial request will only take a few seconds so you can use Google Cloud Function.

When the operation finishes an hour later, you process the output json files by setting up a trigger on your output GCS bucket.

Brendan
  • 1,017
  • 5
  • 7