test_typing.test_awaitable raises a StopIteration with the DLR based async implementation

[slozier]

test_typing.test_awaitable (from the 3.6 test suite) raises a StopIteration with the DLR based async implementation. Here's a simplified version of the test. It should run to completion, but currently produces a StopIteration error (on .NET 8 and .NET 10).

class AwaitableWrapper:
    def __await__(self):
        yield
        return 42

def test_awaitable():
    async def foo():
        return await AwaitableWrapper()
    g = foo()
    g.send(None)

test_awaitable()

@BCSharp

[BCSharp]

The reason the test raises StopIteration rather than run to completion is because awaiting AwaitableWrapper in foo run AwaitableWrapper to completion: bare yield is already a completed task so it does not suspend.

This test exposes one of the two fundamental differences between Python's pull-based coroutine and .NET push-based Task (the second being async function call creating a cold coroutine in Python, vs. a hot Task in C#). This is intentional; under FEATURE_NET_ASYNC the mechanism uses .NET Task, rather than emulating CPython's coroutine mechanism. The Python's mechanism is generic but in practice it is just being consumed by asyncio (and a few less used alternative libraries). So hand-feeding coroutines will work differently under FEATURE_NET_ASYNC, and I did not plan to make it the same. This is also the reason why pure-Python asyncio will likely not work correctly under FEATURE_NET_ASYNC, but our new _asyncio in C# might work just great. Future CPython versions also obsolete pure-Python asyncio in favour of _asyncio.

So, besides asyncio and a few lesser know alternatives, only test code uses it this way. Perhaps it is possible to tweak the implementation a bit so that this test "passes", but in the end we will just be making CPython's coroutine test suite happy, with little practical benefit. There is some work left to be done (see: TODO at Coroutine.cs:71), but to implement it 100% exact as CPython, the code has to emulate CPython coroutine behaviour 100%, which is not what FEATURE_NET_ASYNC is about. That was the alternative code approach (without FEATURE_NET_ASYNC).

[slozier]

Is this more of the same issue? Throws a:

TypeError: object of type 'str' can't be used in 'await' expression

import types

def gen(fut):
    result = yield fut
    return result * 2

@types.coroutine
def foo(): return gen('spam')

async def corofunc():
    return await foo() + 100

coro = corofunc()
coro.send(None)

This one is from test_types.CoroutineTests.test_duck_functional_gen.

[BCSharp]

Is this more of the same issue?

Yes, but there is more to it. Note that by replacing:

@types.coroutine
def foo(): return gen('spam')

with

async def foo(): return gen('spam')

you get the same behaviour in CPython and IronPython (by accident, the push/pull incompatibilty is sidestepped), which happens to be: TypeError: unsupported operand type(s) for +: 'generator' and 'int'. So even in the case of CPython it is different than with the decorator.

I think types.coroutine is used mainly (only?) to support asyncio.Future, so this may be a non-issue when _asyncio gets implemented.