3

I have an server side nodejs express app that responds to requests from front end clients. I need to implement a batch job that will run every hour. If I implement the batch job in the same service, does it mean that the service is 'occupied' until the cron job is completed and it will not be able to serve any requests? Should I create a separate service instead that will run the batch job?

sand
  • 137
  • 1
  • 2
  • 9
  • If you run on the same thread as the express app, yes it will block any requests while being active. You can, however, spawn a child process to handle you batch job thus not affecting the main thread. – Sirko Sep 11 '17 at 09:53
  • ok. that makes sense. However, if I have multiple instances of my app, then there will be multiple instances of the batch job as well? i.e. there will be several of the same batch job running - which I don't want. (The batch job is basically reading data form a file and persisting it in my db). Is my understanding correct? – sand Sep 11 '17 at 10:07

2 Answers2

3

If your batch job does not occupy the cpu 100%, then the server will still serve requests. Every time you do async io or wait for timers, there is plenty of time for the express root routine to deal with requests.

I don't know what your job and server are doing, look at it from Separation of Concerns. If the job scheduler and server belong together, then implement them together, otherwise I recommend to make two services out of it.

CFrei
  • 3,552
  • 1
  • 15
  • 29
  • My service would use the data the batch job puts in it's db. I was under the impression that since node is single threaded, until the batch/cron job is over, the processor will be occupied with it. – sand Sep 11 '17 at 10:02
  • For example have a look here: https://stackoverflow.com/questions/17959663/why-is-node-js-single-threaded. Single threaded - specially with the async/await in ES6 - is imho much faster in development than multi threaded. – CFrei Sep 11 '17 at 10:19
0

Have worked with a lot batch jobs using JAVA programming language and have processed and moved Millions of rows data as ETL tool. And have experience with Core NODE framework with Linux crone schedulers Its works much more faster then previous JAVA operations with low level servers while with java have used heavy configuration server still was facing memory issues.

You can implement crone jobs using NODEJS you have to make SH files and command with node to run JS files directly and call your DBs connection or CSVs to load or transfer.

harnesh patel
  • 1