20

The greenlet package is used by gevent and eventlet for asynchronous IO. It is written as a C-extension and therefore doesn't work with Jython or IronPython. If performance is of no concern, what is the easiest approach to implementing the greenlet API in pure Python.

A simple example:

def test1():
    print 12
    gr2.switch()
    print 34

def test2():
    print 56
    gr1.switch()
    print 78

gr1 = greenlet(test1)
gr2 = greenlet(test2)
gr1.switch()

Should print 12, 56, 34 (and not 78).

Tristan
  • 6,776
  • 5
  • 40
  • 63
  • IronPython and Jython run on VMs that are fully threaded and have their own async IO calls - wouldn't you use those? – mmmmmm May 30 '10 at 18:44
  • Ultimately yes, but I was thinking about writing a pure python version before adding the VM specific versions. This form of flow control is not completely intuitive. – Tristan May 30 '10 at 18:46
  • According to a comment on one of the answers, your ultimate goal is to use `eventlet` in IronPython or Jython. That won't work—not because of the `greenlet`s, but because of `libevent`, a C library that `eventlet` wraps up and depends on for its event loop and reactor. You could conceivably reimplement the whole `libevent` API on top of a native .NET or Java event loop (at least if you don't care about performance, as you say you don't), but that's a whole lot of work. – abarnert Jan 24 '13 at 02:11

2 Answers2

12

This kind of thing can be achieved with co-routines which have been built-in to the standard Python distribution since version 2.5. If IronPython and co are fully compliant with all Python 2.5 features (I believe they are) you should be able to use this idiom.

See this post for more information on how they can be used :) Specifically, you'll be interested in the PDF where the author builds a system using nothing but pure Python that provides similar capabilities to either stackless Python or the Greenlet module.

You may also want to look either Gogen or Kamelia for ideas: these projects both have pure python coroutine implementations which you could either adopt or use as a reference for your own implementation. Take a look at this page for a gentle introduction to the cogen way of doing things.

Note there are some differences between the co-routine implementations here and the greenletimplementation. The pure python implementations all use some kind of external scheduler but the idea is essentially the same: they provide you with a way to run lightweight, co-operative tasks without the need to resort to threads. Additionally both the frameworks linked to above are geared towards asynchronous IO very much like greenlet itself.

Here's the example you posted but rewritten using cogen:

from cogen.core.coroutines import coroutine
from cogen.core.schedulers import Scheduler
from cogen.core import events

@coroutine
def test1():
    print 12
    yield events.AddCoro(test2)
    yield events.WaitForSignal(test1)
    print 34

@coroutine
def test2():
    print 56
    yield events.Signal(test1)
    yield events.WaitForSignal(test2)
    print 78

sched = Scheduler()
sched.add(test1)
sched.run()

>>> 12
>>> 56
>>> 34

It's a little more explicit than the greenlet version (for example using WaitForSignal to explicitly create a resume point) but you should get the general idea.

edit: I just confirmed that this works using jython

KidA% jython test.py 
12
56
34
jkp
  • 78,960
  • 28
  • 103
  • 104
  • 1
    This is a nice example, however I'm really interesting in using exactly the greenlet api, without the yield statements. I'd like to enable eventlet, which requires greenlet, on Jython and IronPython. – Tristan Jun 07 '10 at 22:52
  • @tristan: if you want to emulate something just like greenlet you'd have to take these ideas and wrap them in an identical API. It should be possible to come up with something close. I can have a go and showing you something like this but I think you need to take whats been shown and run with it! Clearly greenlet itself is not available, but this shows that the principles carry over and are possible to reuse. – jkp Jun 08 '10 at 07:42
  • There's a spelling error in your post. It should be `Cogen` instead of `Gogen`. – suzanshakya Sep 30 '13 at 08:37
10

It's not possible to implement greenlet in pure Python.

UPDATE:

  • faking greenlet API with threads could be indeed doable, even if completely useless for all practical purposes
  • generators cannot be used for this as they only save the state of a single frame. Greenlets save the whole stack. This means gevent can use any protocol implemented on top of the standard socket (e.g. httplib and urllib2 modules). Generator-based frameworks require generators in all layers of your software, so httplib and tons of other packages are thrown away.
Denis
  • 3,760
  • 25
  • 22
  • 4
    Not possible or not performant? Can we fake the greenlet API with threads? – Tristan May 31 '10 at 02:56
  • 1
    Actually, that's an interesting point. Maybe we could indeed fake greenlet API with threads. However, it's not practical, as in that case gevent and eventlet will provide you the API that you already have (like sockets), only with more overhead. – Denis Jun 01 '10 at 01:37
  • 3
    Pure python version can be possible only with threads. But I think, it is more useful to provide VM-specific greenlet implementation for IronPython, Jython — maybe this would help adopting greenlet as standart module for Python. – andreypopp Jun 03 '10 at 16:51
  • 1
    He's right about there being a difference faking this kind of stuff with generators: you don't get a full-stack which is a pretty major difference. – jkp Sep 15 '11 at 19:12
  • Yeah. This answer is completely true. "Just use generators" is becoming a weak response to having true coroutines. Generators are great, but they're not full blown coroutines and require changing your code to fit. – Matt Joiner Sep 22 '11 at 00:38