1

Here're goals i'm trying to achieve:

  • Take the scheduled jobs out of microservice because it can and would harm timings/performance
  • Execute jobs in a separate computation cluster aka workers
  • Avoid code duplication: i want to keep all my business logic in one Service, all DB-related operations in one Dao, do not write additional services/daos for jobs
  • Avoid dependency management problems: different jobs may require different libs/versions/etc. For instance, job from ServiceA may use javax.annotation-api while job originated from ServiceB may use jakarta.annotation-api. Making a worker depend both on ServiceA and ServiceB will cause build or runtime problems.

Are there any approaches/libraries/solutions to achieve all the goals at the same time?

UPD: Both Temporal.io and quartz are not quite what I need - they both require worker to depend on workflow tasks. I can imagine that I’m approaching the issue I face in incorrect way, so architectural advises are also appreciated

Seventh
  • 107
  • 5

2 Answers2

1

From my perspective you have the option to use one of three possible solutions:

  1. Most straight forward - Ensure that service logic which is required in the jobs also implements a local API (programming API).

As such it can act as and be imported as a library and reused in jobs without code duplication.

If you have a larger development organization you also want to make sure that such libraries are correctly version managed and version releases are pre-planned, which allows the teams using the libraries to treat them like they would third party libraries.

Also there is no magic - You would have to work through any build/dependency problems if there would be conflicts. (Since your question sounds like this is a deal breaker, let's take a look at the other solutions.)

  1. The second solution would be to provide a wrapper for each service logic that allows to access functionality via CLI. That means you don't have to import the libraries, but rather execute them as jars/executables through the CLI. This would allow you to use the same code but avoid dependency problems.

(You will still have to deal with version management and version upgrades, etc.)

  1. In case you use containerized deployments/hosting you can also consider to bundle up multiple containers together just for your jobs, where each job gets its own private service container instances for use during the job. Kubernetes and Docker Compose for example have options to run such multi-container deployments/jobs.

That solution would allow you to reuse the same services as they run for other purposes, but you have to make sure that they are configurable enough to work in this scenario.

One problem that all of the approaches have is that you have to make sure there are no runtime conflicts between your jobs and the deployed regular services. (For example state conflicts)

In terms of how to execute jobs it will depend on your deployment scenario. Kubernetes has an option to run containers as jobs natively, which makes it easy to bundle multiple jars, etc. But it is always an option to deploy a dedicated scheduler or workflow tool like Apache Airflow to run your jobs.

Oswin Noetzelmann
  • 9,166
  • 1
  • 33
  • 46
1

From architectural perspective, expose service (business logic) via API. Have schedulers run on separate instance or if you are using some of the popular cloud solutions have their FaaS (function as a service in your case scheduler) trigger service API via HTTP (any or dedicated instance).

  • Azure -> user azure functions
  • AWS -> lambda functions
  • Google Cloud -> Google Cloud Functions

All of the above have comprehensive guide how to create scheduled function aka trigger.

Hope this helps and I'm not off topic.

doom4s
  • 141
  • 6