8

I'm looking for non-trivial resources on concepts of asychronous programming, preferably books but also substantial articles or papers. This is not about the simple examples like passing a callback to an event listener in GUI programming, or having producer-consumer decoupled over a queue, or writing an onload handler for your HTML (although all those are valid). It's about the kind of problems the lighttpd developers might be concerned with, or someone doing substantial business logic in JavaScript that runs in a browser or on node.js. It's about situations where you need to pass a callback to a callback to a callback ... about complex asynchronous control-flows, and staying sane at the same time. I'm looking for concepts that allow you to do this systematically, to reason about this kind of control-flows, to seriously manage a significant amount of logic distributed in deeply nested callbacks, with all its ensuing issues of timing, synchronization, binding of values, passing of contexts, etc.

I wouldn't shrink away from some abstract explorations like continuation-passing-style, linear logic or temporal reasoning. Posts like this seem to go into the right direction, but discuss specific issues rather than a complete theory (E.g. the post mentions the "reactor" pattern, which seems relevant, without describing it).

Thanks.

EDIT:

To give more details about the aspects I'm interested in. I'm interested in a disciplined approach to asynchronous programming, a theory if you will, maybe just a set of specific patterns that I can pass to fellow programmers and say "This is the way we do asynchronous programming" in non-trivial scenarios. I need a theory to disentangle layers of callbacks that randomly fail to work, or produce spurious results. I want an approach which allows me to say "If we do it this way, we can be sure that ...". - Does this make things clearer?

EDIT 2:

As feedback indicates a dependency on the programming language: This will be JavaScript, but maybe it's enough to assume a language that allows higher-order functions.

EDIT 3:

Changed the title to be more specific (although I think design patterns are only one way to look at it; but at least it gives a better direction).

ThomasH
  • 22,276
  • 13
  • 61
  • 62
  • krondo (blog) is very good: http://krondo.com/blog/?page_id=1327 – ypercubeᵀᴹ May 09 '11 at 12:40
  • Various other links related to Twisted: http://stackoverflow.com/questions/1888139/python-twisted-where-to-start – ypercubeᵀᴹ May 09 '11 at 12:41
  • @ypercube Thanks. I wouldn't refrain from studying a product (like Twisted), but am still hoping to find something more general. – ThomasH May 09 '11 at 12:49
  • This is similar to learning recursion. You don't need to start with a large project, if you understand the Tower of Hanoi problem then you can extend it. If you can have one function with a callback call another one with a callback, then you can continue, but, the trick is how to get the functions defined so you can have multiple layers. At that point currying is a concept you may want to get familiar with. – James Black May 09 '11 at 12:50
  • Asynchronous programming is a very broad area, so which aspect are you looking at? – James Black May 09 '11 at 12:52
  • @James Can you put your currying suggestion into an answer?! Then it would be easier to discuss. – ThomasH May 09 '11 at 13:14
  • 1
    You may want to just look at http://stackoverflow.com/questions/2911822/javascript-execute-a-bunch-of-asynchronous-method-with-one-callback or google "javascript callback design pattern" to get more ideas, but if you design a generic library for this, then just tell people to use the library, and that is how callbacks are handled at your company. – James Black May 09 '11 at 13:46
  • @James Great, again. If you put it in an answer I could upvote it :). I was probably searching with the wrong terms ("asynchronous", "event-driven"). As for the library, we're using it directly in the project, but maybe it can be made into an API on its own. Never thought this could be possible. – ThomasH May 09 '11 at 13:54
  • Related http://stackoverflow.com/q/5022883/127465 – ThomasH May 09 '11 at 14:37
  • Related http://stackoverflow.com/q/5483441/127465 – ThomasH May 09 '11 at 14:39

1 Answers1

3

When doing layered callbacks currying is a useful technique.

For more on this you can look at http://en.wikibooks.org/wiki/Haskell/Higher-order_functions_and_Currying and for javascript you can look at http://www.svendtofte.com/code/curried_javascript/.

Basically, if you have multiple layers of callbacks, rather than having one massive parameter list, you can build it up incrementally, so that when you are in a loop calling your function, the various callback functions have already been defined, and passed.

This isn't meant as a complete answer to the question, but I was asked to put this part into an answer, so I did.

After a quick search here is a blog where he shows using currying with callbacks:

http://bjouhier.wordpress.com/2011/04/04/currying-the-callback-or-the-essence-of-futures/

UPDATE:

After reading the edit to the original question, to see design patterns for asynchronous programming, this may be a good diagram: http://www1.cse.wustl.edu/~schmidt/patterns-ace.html, but there is much more to good asynchronous design, as first-order functions will enable this to be simplified, but, if you are using the MPI library and Fortran then you will have different implementations.

How you approach the design is affected heavily by the language and the technologies involved, that any answer will fall short of being complete.

James Black
  • 41,583
  • 10
  • 86
  • 166
  • Thanks, James. I will think about applying currying. I've also edited the question to include the main implementation language, though I would assume the basic principles are the same for languages that allow passing and returning of functions ("callbacks"), although it might be agreeably more convenient in one over the other. – ThomasH May 09 '11 at 13:43