0

I am using Cloud Run to simplify some of my batch processing on very large images. I break down each task to one single image, however, some of the images can get as large as 5 GB and the operation can take up to 45 minutes. Cloud Run has a limit of 900 seconds and Pub/Sub with push subscription has a limit of 600 seconds. Is there a way to extend these deadlines? I understand that push subscription doesn't have the "extend deadline" option of a pull subscription. What is a decent workaround? How about the 900s limit?

If this doesn't work out I will switch back to k8s where I have far more control over these but I have been enjoying the simplicity of Cloud Run and would like to make it work. I appreciate any feedback.

Dustin Ingram
  • 20,502
  • 7
  • 59
  • 82
Mohammad S
  • 61
  • 1
  • 3
  • 1
    Can you save your progress and resume it in another task? Or break down processing one image into multiple tasks? – Cully Jun 09 '20 at 06:24
  • 1
    Cloud Task have a timeout of 1800 seconds. Cloud Run timeout will be extended in the future. I can't say if it will be in 2020. – guillaume blaquiere Jun 09 '20 at 07:31
  • Yes, that's what I ended up doing for this task. Saving the progress and resume in another container once it is terminated. Not ideal but there was a way for this task. That's not necessarily the case for more complex or non-deterministic tasks. I look forward to this extension. Thank you for your response – Mohammad S Jun 15 '20 at 16:58

0 Answers0