Python coroutines look magical from the outside. You write async def, you await something, and somehow execution suspends and resumes. But there is no magic. Everything is implemented in CPython as an extension of generator objects — with a handful of bytecode instructions and a frame evaluation mechanism. This post traces the execution of a coroutine step by step from Python bytecode down through the CPython C source.
Coroutines Are Generators With Extra Rules
In CPython, a coroutine object (type PyCoroObject) shares almost all its implementation with a generator object (PyGenObject). Both are defined in Objects/genobject.c. The critical difference is that coroutines enforce two rules: they cannot be iterated with __iter__ (you cannot use for x in coro()), and they must be awaited.
When you define async def f(): …, the compiler emits a code object with the CO_COROUTINE flag set in co_flags. When Python calls f(), it checks this flag and builds a coroutine object instead of a generator. But internally, both are just a Python frame suspended at a particular bytecode offset.
The Frame Object: Where State Lives
Every coroutine (and generator) holds a reference to a frame object (_PyFrameObject). The frame stores the local variables, the evaluation stack, and the last_i field — the index of the last bytecode instruction executed. When execution suspends (at a yield or await), the frame is kept alive with last_i pointing at the suspension point.
When the coroutine is resumed, the frame evaluator (ceval.c) restores the stack and jumps directly to last_i + 2 (the next instruction). This is why coroutines are cheap to suspend and resume — there is no OS thread context switch, only a frame pointer update.
SEND: The Bytecode Instruction That Drives It All
In Python 3.12+, the SEND bytecode instruction is the mechanism by which await passes control. When you write await some_future, the compiler generates code roughly equivalent to: push the awaitable onto the stack, enter a SEND loop until StopIteration is raised.
SEND calls gen_send_ex() in genobject.c. This function: sets frame.f_back to the calling frame, pushes the sent value onto the frame’s stack, calls _PyEval_EvalFrameDefault() (the main eval loop) with the resumed frame, and handles the return. If the inner frame hits a YIELD_VALUE instruction, execution returns to the caller with the yielded value. If the inner frame raises StopIteration, SEND catches it and extracts the return value.
/* Simplified from Objects/genobject.c */ static PyObject * gen_send_ex(PyGenObject *gen, PyObject *arg, int exc) { PyFrameObject *f = gen->gi_frame; /* Resume from last_i */ f->f_stacktop = …; /* restore stack */ result = _PyEval_EvalFrameDefault(f, exc); if (result) { /* YIELD_VALUE hit: result is the yielded value */ return result; } else if (PyErr_ExceptionMatches(PyExc_StopIteration)) { /* Coroutine returned: extract return value */ … } }
→ Related: Python asyncio event loop internals — how Tasks call send() (Blog 01)
What send(value) Does Differently Than __next__
Calling coro.send(None) is identical to calling next(coro). The difference is coro.send(value) where value is not None — the sent value becomes the result of the yield expression inside the coroutine. For regular coroutines awaiting Futures, the value sent back is always the Future’s result (or exception). asyncio’s Task.__step() sends the result back using coro.send(result) after the awaited Future resolves.
The value is placed at the top of the frame’s stack at the point of suspension, so when evaluation resumes, the next instruction sees it as the result of the yield.
throw() and Exception Propagation
gen.throw(exc_type, exc_val, exc_tb) injects an exception into the coroutine at its current suspension point. gen_send_ex() is called with exc=1, which causes _PyEval_EvalFrameDefault() to raise the exception at last_i rather than executing the next instruction normally. If the coroutine has a try/except block covering that point, it handles the exception. Otherwise, the exception propagates out and the coroutine is finalized.
This is how asyncio propagates CancelledError into tasks. Task.cancel() calls coro.throw(CancelledError), which surfaces inside the coroutine at whatever await point it is suspended on.
CO_ITERABLE_COROUTINE and @types.coroutine
Generator-based coroutines (pre-async/await) used the @asyncio.coroutine decorator (now removed) or @types.coroutine. The latter sets the CO_ITERABLE_COROUTINE flag, which allows yield from to delegate to the generator and makes it compatible with await. This is why older asyncio code using yield from still works with modern asyncio — same SEND mechanism, different flags.
→ Related: Python metaclass deep dive — reading type.__new__ in CPython (Blog 03)
Finalization: What Happens When You Don’t Await
If a coroutine object is garbage collected without being awaited, CPython emits a RuntimeWarning: coroutine ‘f’ was never awaited. This warning is generated in gen_dealloc() in genobject.c. If the coroutine had started executing (f_stacktop != NULL), the GC calls gen_close(), which throws GeneratorExit into the coroutine to let it clean up.
This is why unawaited coroutines that open resources can leak — GeneratorExit causes finally blocks to run, but only if the coroutine had started.
Conclusion
Python coroutines are frames with a saved instruction pointer, driven by send() and throw(), built on the same gen_send_ex() machinery as generators. The CO_COROUTINE flag, the SEND bytecode, and the frame evaluator are the three pieces. Reading Objects/genobject.c and Python/ceval.c with this map in mind will answer every remaining question about how async Python behaves at runtime.
Leave a Reply