2

I am writing a distributed system in which jobs (processes) will be run on multiple nodes (LINUX machines).

I want to be able to launch job on these machines, terminate a job (kill), monitor what is running on each machine.

I am looking for a framework (preferably in Java) which I can use to implement this solution.

I have looked at using the underlying components that frameworks like Hadoop or Akka use but have not come up with anything satisfactory.

A few additional constraints on what I'm looking for:

  1. It should have reasonable proliferation into production grade projects
  2. It should allow me to launch UNIX processes written in arbitrary languages
  3. It should be open source
  4. It should be lightweight -- An answerer has suggested I look at agent based frameworks -- though these satisfy the requirements I've posted -- I've found them to be very heavy weight. I'm looking for something that really fulfills the stated requirements and does not require me to adopt a more comprehensive methodology.

Thanks.

user1172468
  • 5,306
  • 6
  • 35
  • 62
  • if you're happy with supervisord why not use that to run your java programs? – Arnon Rotem-Gal-Oz Feb 28 '12 at 04:11
  • Well, I am looking for a framework where I can add my own rules -- these rules would be like a strategy of how many process to start, when to start them, etc. I don't think supervisord has that type of flexibility or was geared for that type of a use case. – user1172468 Feb 28 '12 at 10:37

1 Answers1

2

It's called multi-agent system and there are several frameworks available that you can use:

Łukasz Rżanek
  • 5,786
  • 27
  • 37
  • Hi and many thanks for your response. I am aware of Agent based frameworks that would essence fulfill the requirements I outlined. However I'm looking for something that is very light weight and focused on solving the stated problem -- a distributed supervisord in essence. – user1172468 Feb 27 '12 at 19:22