Can we spin off a kubernetes cronjob automatically and dynamically? How can we do it in AWS EKS, Azure AKS based on queues or notifications?

Question

For my microservice based application, I am designing a component which is as follows:

1. Task that we want to execute is of periodic nature. For it, i planned to make use of the Kubernetes cron-jobs. It executes the job every 1 hour. This works perfectly fine.
2. In few scenarios, i want to execute this task on-demand (in stead of waiting for next hour window). For example, if next job time is 2:00pm, i want to execute it early, say 1:20pm.

There is a related question - How can I trigger a Kubernetes Scheduled Job manually?

But I am not looking for a manual way of achieving it or explicitly calling kubectl commands. Is there a way do it automatically, based on events/queues?

Our application is deployed on AWS EKS and Azure AKS. Can I integrate the k8 clusters to read onto some queues/pub-subs (ex. aws-sqs, aws-sns) and do it dynamically?

Your help would be immensely appreciated!

Answer 1

If you application is running on Kubernetes and don't want to get migrated to serverless function and keep everything inside the Kubernetes cluster you can use the Knative.

Scale to Zero With Knative

Knative is a serverless platform that is built on top of Kubernetes. It provides higher-level abstractions for common application use cases.

One key feature is its ability to run generic (micro) service-based applications as serverless with the help of built-in scale to zero support. Knative has introduced its own autoscaler, Knative Pod Autoscaler (KPA), that supports scale to zero for any service that uses non-CPU-based scaling matrics.

update your micro service to running with Knative minor change will be there and you can run it on Kubernetes.