6

In implementing coroutines in the Transcrypt Python to JavaScript compiler, I have the following weird problem.

Transcrypt uses the native parser of CPython 3.6 to generate an AST. For async global function defs it generates an AsyncFunctionDef node. But for async methods it doesn't! Nevertheless CPython itself seems to compile async methods corrrectly.

So the following piece of code runs with CPython, but Transcrypt cannot run it because the AST generated by CPython's AST module seems to lack the AsyncFunctionDef node for methods (as opposed to global functions).

So the following piece of code does NOT generate an AsyncFunctionDef node:

class C:
    def __init__ (self):
        self.aTime = 2

    async def g (self, waw, asio):
        print ('g0')
        await waw (self.aTime, asio)
        print ('g1')

What am I missing? Async methods are officially supported, aren't they? Couldn't find anything specific in PEP 492.

The complete code of the example is:

from org.transcrypt.stubs.browser import __pragma__, __envir__

# Note that CPython will ignore all pragma's



# Provide waitAWhile for Transcrypt

__pragma__ ('js', '{}', '''
    function waitAWhile (aTime, asio) {
      return new Promise (resolve => {
        setTimeout (() => {
          resolve (aTime);
        }, 1000 * aTime);
      });
    }
''')



# Provide waitAWhile for CPython

__pragma__ ('skip') # Compile time, needed because import is done compile time

import asyncio

def waitAWhile (aTime, asio):
    return asio.sleep (aTime)

__pragma__ ('noskip')



# Actual code to be tested    

async def f (waw, asio):
    print ('f0')
    await waw (2, asio)
    print ('f1')

class C:
    def __init__ (self):
        self.aTime = 2

    async def g (self, waw, asio):
        print ('g0')
        await waw (self.aTime, asio)
        print ('g1')

c = C ()


# Just call async functions for Transcrypt, since in the browser JavaScript is event driven by default

if __envir__.executor_name == __envir__.transpiler_name:
    f (waitAWhile, None)
    c.g (waitAWhile, None)
    c.g (waitAWhile, None)
    f (waitAWhile, None)



# Create event loop and tasks for CPython, since it isn't event driven by default

else:
    eventLoop = asyncio.get_event_loop ()
    tasks = [
        eventLoop.create_task (f (waitAWhile, asyncio)),
        eventLoop.create_task (c.g (waitAWhile, asyncio)),
        eventLoop.create_task (c.g (waitAWhile, asyncio)),
        eventLoop.create_task (f (waitAWhile, asyncio)),
    ]

    waitingTasks = asyncio.wait (tasks)
    eventLoop.run_until_complete (waitingTasks)
    eventLoop.close ()
Jacques de Hooge
  • 6,750
  • 2
  • 28
  • 45

1 Answers1

1

Eventually I got the parser to work properly. I must have blocked the parse somewhere else initially. Probably I forgot to call visit from a node higher up the tree. Unfortunaly I can't reproduce the problem anymore.

For those interested, the parser code is at:

https://github.com/QQuick/Transcrypt/blob/master/transcrypt/modules/org/transcrypt/compiler.py

line 2045.

Most important: Python's ast module works fine, although it could do with a bit more documentation.

There's a (quite compact but useable) 3rd party doc at:

https://greentreesnakes.readthedocs.io/en/latest/

Jacques de Hooge
  • 6,750
  • 2
  • 28
  • 45
  • Manually calling self.visit is going to be difficult to maintain, you're better off calling self.generic_visit(node) at the end of your overridden methods – anthony sottile Aug 05 '17 at 12:22
  • I am not sure I understand you correctly. The docs say: `generic_visit(node)` This visitor calls `visit()` on all children of the node. But in many cases I don't want to visit all children. – Jacques de Hooge Aug 05 '17 at 12:37
  • Having written a ton of `NodeVisitor`s, usually "not wanting to visit all children" is the special case, and in that case you can just early `return` (None). Most of the bugs I've found when writing `NodeVisitor`s are exactly as described in this question and due to missing `visit` / `generic_visit` -- a simple best practice I've been sticking to is always having `self.generic_visit(node)` as the last line of any of my `visit_*` methods. – anthony sottile Aug 05 '17 at 14:43
  • Thanks, I'll keep this in mind when working on the parser! – Jacques de Hooge Aug 05 '17 at 16:35