12

I'm surveying the google cloud tools that could be used to: deploy and update a micro-service shaped application. So far I focused my attention on two solutions: (a) Container clusters; (b) Managed Instance groups plus autoscaler. Could you please help me to decide which way I should go. You'll find below some details and requirements:

  • The application PULLs tasks from a pubsub topics and write results to a another pubsub topic;
  • Tasks are independent from each other;
  • The number of worker should autoscale wrt. the CPU usage level;
  • Each worker uses up to 10GiB of RAM.
  • At startup time a worker needs several minutes (<=5mn) to be ready to process tasks;
  • Out of the box rolling update is a plus;
  • Workers share a memcache sever, except that, there is strictly none communication of whatsoever kind between workers;
  • I suspect there is no need for load balancing, since a worker will process a new task as soon as it can;
  • Logs are pushed to a collection API (google cloud logging or third party).

I did a MWE for solution (a) and solution (b). So far I have the sentiment that I won't use the kubernetes features. Hence I'm more inclined towards solution (b).

What do you think ?

Bests, François.

Grzenio
  • 35,875
  • 47
  • 158
  • 240
François Kawala
  • 436
  • 3
  • 18
  • 3
    If managed instance groups meets your current needs, it's easier to start with; but investing in Kubernetes is good for the long term, eventually you'll need the features, maybe in this project or others. – Dagang Oct 26 '16 at 16:15
  • I'm in the same boat -- each task is long-running (minutes) and heavyweight (lots of CPU and RAM). Pulling tasks from a PubSub topic, just like you. What I *really* want is a solution that autoscales instances based on load (down to zero when inactive) but nothing does this as far as I can tell. (Maybe App Engine Flexible??) – GaryO Dec 05 '19 at 20:23
  • @GaryO - This can easily be achieved with managed instance groups, which is exactly what we do. You can configure instance groups to autoscale based on the number of unacknowledged messages in a pubsub topic: https://cloud.google.com/compute/docs/autoscaler/scaling-stackdriver-monitoring-metrics#example_using_instance_assignment_to_scale_based_on_a_queue – Josh Bothun May 09 '20 at 20:02

1 Answers1

12

I would say that the main difference between hosted Kubernetes and Managed Instance Groups [MIGs] is that Kubernetes operates on the abstraction level of Containers and MIGs operate on VM instances. So it is easier for you to package your software into containers, then go for Kubernetes, if it is easier to package you software into an image then use MIGs.

Grzenio
  • 35,875
  • 47
  • 158
  • 240