20

I'm looking for tools that implement the distributed agent/service model ... I guess we could also call this grid or cloud computing, but I'm not sure the term is exactly analagous. A distributed agent model would be one where developers build agents that are dispatched to a framework and the framework executes the agent somewhere in the distributed environment.

Specifically, I'm looking for a framework that support dynamic distribution across the grid/cloud, but replaces the transient agent with a more persistent service. The goal here is to minimize the amount of time and effort it requires to deploy a service into the grid/cloud container.

FWIW, I think Digipede gets pretty close to this, but I'm interested in what else is out there. I'm under the impression that assembly distribution would have to be considered in order for the solution to be viable.

Ajaxx
  • 2,500
  • 4
  • 27
  • 38
  • 1
    Too bad this was closed. Try these. [MPAPI](http://mpapi.codeplex.com/) [MPI.NET](http://osl.iu.edu/research/mpi.net/) [MpiVisor](https://github.com/duovia/duovia) I am affiliated with the latter. – Tyler Jensen Apr 22 '13 at 22:19

5 Answers5

8

There's a project at Microsoft called Dryad. Here's a good podcast with more info: http://www.dotnetrocks.com/default.aspx?showNum=378 It uses LINQ and is supposed to be easy to use. Microsoft has it doing some production stuff for Microsoft's AdCenter. Here's the research page. http://research.microsoft.com/research/sv/Dryad/

Dryad is quite expressive. It completely subsumes other computation frameworks, such as Google's map-reduce, or the relational algebra. Moreover, Dryad handles job creation and management, resource management, job monitoring and visualization, fault tolerance, re-execution, scheduling, and accounting.

P.S. I don't see a download, but if you contact them and you are serious you could probably get it.

Update: it's now available for download here: http://research.microsoft.com/en-us/downloads/03960cab-bb92-4c5c-be23-ce51aee0792c/default.aspx

Booji Boy
  • 4,522
  • 4
  • 40
  • 45
2

This may be a little late to the party - but we're the developers of Alchemi. We've developed a better version of the framework to address its limitations.

You might want to check out the Utilify Platform (www.utilify.com) - which is conceptually similar to Alchemi - with a more flexible model.

You do need a grid manager, and a database - however, you could use the free SQL Server Express version and it is tested to work on Windows Vista, XP, 2003.

We're happy to help you in using Utilify for your application.

Regards, Krishna

Krishna
  • 2,997
  • 1
  • 23
  • 31
1

Qizmt is used by MySpace for distributed MapReduce operations. It's analogous to Hadoop from the Java world. Both projects are based on MapReduce by google. It runs on normal Windows boxes, unlike dryad which requires Windows Server High Performacne Computing (HPC)

Plus it's open source. Which is nice. http://code.google.com/p/qizmt/

pnewhook
  • 4,048
  • 2
  • 31
  • 49
0

I've been looking into a similar thing for my own work. So far I've found Digipede, Alchemi and NGrid, but they all need a SQL database as a grid manager except for NGrid which isn't as mature.

Cameron MacFarland
  • 70,676
  • 20
  • 104
  • 133
  • Hi Cameron, is there a particular reason why you see that having a grid manager is a problem? Would you prefer that the scheduling, failure management, communication to a number of machines etc are all handled on the client? - Krishna. – Krishna Mar 09 '10 at 05:51
  • No, I'd just prefer a manager that doesn't use SQL. – Cameron MacFarland Mar 09 '10 at 06:50
0

Voyager from Recursion Software provides a mature framework for many styles of distributed architectures.

Todd Stout
  • 3,687
  • 3
  • 24
  • 29
  • Is voyager in anyway related to a product called Voyager that was created by ObjectSpace about a decade ago. That was a Java-based agent framework. – Ajaxx Aug 06 '09 at 18:46
  • Yes, it is the same, updated/maintained of course. Recursion purchased the rights to Voyager from ObjectSpace before ObjectSpace became defunct. Voyager is available for both java and .Net. – Todd Stout Aug 19 '09 at 18:49