0

i'm using from several time Hangfire with SQL and i don't have any problem.

Classic use cases that i will have are:

  1. From a web app, user enqueue an operation and can check all pending operations. Where operation ends he has a notification via SignalR and can check output of a job. An example is the same Azure Portal, where every operation is enqueued and status is visible in the bell on top right.
  2. Recurring job that runs every few seconds to perform scheduled operations indipendent from user's interaction. Jobs must have a lock concept, so until one is running, nothing else of the same type should start for that user.

Recently i'm moving to Azure, using k8s for some microservices. I think that i will use a microservice with Hangfire, do you thiks that Hangfire and microservices can coexist?

If i don't use Hangfire i have to replicate something similar using Azure Function (high cost), manage a storage for job's state, so will worth it using something different?

How do you manage these scenario in microservice architecture? Bye

I'm working on running Hangfire in a dockerized microservice.

Andrea92
  • 1
  • 1
  • I found this answer here I find useful possibly to your question https://stackoverflow.com/a/54029417/3225 – kenny Jul 08 '23 at 14:32
  • Azure has scheduling services already that go beyond Hangfire. You're using Hangfire to cover multiple scenarios for which there are better open source options. Data processing isn't the same as queuing backend jobs for users, running tasks on a schedule *or* executing a flow of tasks. There's [Azure Scheduler](https://azure.microsoft.com/en-us/pricing/details/scheduler/) for generic scheduling, [Azure Data Factory](https://learn.microsoft.com/en-us/azure/data-factory/) for data which includes [Managed Airflow](https://learn.microsoft.com/en-us/azure/data-factory/concept-managed-airflow) – Panagiotis Kanavos Jul 10 '23 at 10:10
  • You could also host [Airflow](https://airflow.apache.org/docs/apache-airflow/stable/index.html) or [Dagster](https://dagster.io/) in your Kubernetes cluster to avoid Azure fees. For more complex workflows, there are Azure Logic Apps and Power Apps – Panagiotis Kanavos Jul 10 '23 at 10:12
  • You need to consider how you actually use Hangfire and how much each option will cost. In Azure, you're paying for everything. Even if you choose the "free" edition of Hangfire you're still paying for the VM and either SQL Server or Redis, either as a managed service subscription, or as a service you host yourself on a VM. – Panagiotis Kanavos Jul 10 '23 at 10:18

1 Answers1

0

Since you are using Kubernetes on the server you can use CronJobs. https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/

The upside to this instead of using Hangfire is that you have a pod for each actual job that is executed.

There is also no extra pod that is running just to support hangfire itself as the K8 is managing that on its own.

Additionally there are certain patterns can be used to have parallel execution for your jobs and maximizing the usage of K8 https://kubernetes.io/docs/concepts/workloads/controllers/job/#job-patterns

The downsides: Hangfire has a nice dashboard which you will have to replace with something like grafana or any other dashboard.

Hangfire has the job details and state in the database - will have to adjust your jobs to work stateless and only with the cronjob config files and logs.

misha130
  • 5,457
  • 2
  • 29
  • 51
  • The dashboard isn't much of an upside once you compare it to Airflow or other open source options. The "nice" parts are sold separately too. The OP is using Hangfire for much more than cron jobs too, for which Hangfire isn't a great option to begin with. – Panagiotis Kanavos Jul 10 '23 at 10:20
  • Yea the state persistence in the database. I understand but as I wrote migrating to stateless is an option, for instance I migrated from hangfire to quartz which forced me to do so too. – misha130 Jul 10 '23 at 11:31