11

I went to the SF Python meetup when Guido talked about Tulip, the future asyncIO library for asynchronous operations in Python.

The take away is that if you want something to be run asynchronously you can use the "yield from" + expression and a couple of decorators to specify that the call to what comes after yield from should be executed asynchronously. The nice thing about it is that you can read the statements in that function normally (as if it was synchronous) and it will behave as if it was synchronous with respect to the execution of that function (return values and error/exception propagation and handling).

My question is: why not have the opposite behavior, namely, have all function calls be by default async (and without the yield from) and have a different explicit syntax when you want to execute something synchronously?

(besides the need for another keyword/syntax spec)

Piotr Dobrogost
  • 41,292
  • 40
  • 236
  • 366
Gabriel
  • 1,078
  • 1
  • 8
  • 15
  • @iCodez I guess asyncio as the name of the standard library will be used more often than Tulip. Besides Tulip tag is already taken for C++ framework. I tagged all Python questions referring to Tulip with `python-asyncio` (see `python-multithreading` for *prior art*) – Piotr Dobrogost Nov 02 '13 at 12:23
  • try: `sentences = tokenize_sent(your_question); random.shuffle(sentences); text = "\n".join(sentences)` and see how easy it is to understand it or whether the meaning stays the same. async by default scrambles the order: all the problems of preemptive multithreading without the benefits. – jfs May 05 '14 at 11:48
  • here's [code example](https://gist.github.com/11535593) for demonstration. – jfs May 05 '14 at 12:41

2 Answers2

17

The real answer is that Guido likes the fact that asynchronous yield points are explicit in coroutines, because if you don't realize that a call can yield, then that's an invitation to concurrency problems -- like with threads. But if you have to write an explicit yield from, it's fairly easy to make sure it doesn't land in the middle of two critical operations that should appear atomic to the rest of the code.

As he mentions in his PyCon 2013 keynote, there are other Python async frameworks like Gevent, which are async by default, and he doesn't like that approach. (at 11:58):

And unfortunately you're still not completely clear of the problem that the scheduler could at a random moment interrupt your task and switch to a different one. [...] Any function that you call today that you happen to know that it never switches, tomorrow someone could add a logging statement or a lazy caching or a consulting of a settings file. [...]

intgr
  • 19,834
  • 5
  • 59
  • 69
  • But a non-async declared function may spawn a thread and lead to the exact same concurrency issues. I think GvR was wrong in making async a viral notion in the language. – user48956 Jan 15 '20 at 22:42
11

Note that the possible uses of yield from are a small part of the asynch PEP, and never need to be used. Maybe Guido oversold them in his talk ;-)

As to why functions aren't being changed to always be async by default, that's just realism. Asynch gimmicks bring new overheads and semantic complications, and Python isn't going to slow down and complicate life for everyone to make a few applications easier to write.

In short, "practicality beats purity" ;-)

Tim Peters
  • 67,464
  • 13
  • 126
  • 132
  • 1
    Hmmm... What would have been practical would be to provide incremental ways to integrate async into a codebase (e.g. sync(coro) for non-async code to await a coroutine). As it stands if I want to port a flask server to an async one, and use some async database library, every function in the stack between the endpoint handler and DB's socket.read now needs to be modified async-await. And worse, everything that depends on those functions. This is purity over practicality. – user48956 Jan 15 '20 at 22:50