理解Python的协程机制-Yield

根据PEP-0342 Coroutines via Enhanced Generators，原来仅仅用于生成器的yield关键字被扩展，成为Python协程实现的一部分。而之所以使用协程，主要是出于性能的考虑：一个活跃的Python线程大约占据8MB内存，而一个活跃线程只使用1KB不到内存。对于IO密集型的应用，显然轻量化的协程更适用。

概述

原来，yield是一个statement，即和return一样的语句，但是在PEP-0342后，yield statement被改造为了yield expression。其语法如下：

Yield expressions:

1. yield_atom ::= "(" yield_expression ")"
2. yield_expression ::= "yield" [expression_list]

yield表达式只能在函数体内使用，会导致该函数变为一个 生成器函数

当生成器函数被调用，它会返回一个 生成器 ，用以控制生成器函数的执行.

生成器的第一次执行，必须使用gen_func.next()方法，函数会执行到第一条yield的位置并返回该处yield语句之后的表达式值。

接下来，可以交替使用gen_func.next与gen_func.send方法对协程的执行。send方法会传入一个参数，该参数即yield表达式的值，可以在生成器函数里面被接收。而next则不传入参数。当send or next被执行时，生成器函数会从yield表达式处继续执行。直到下一次出现yield语句，再次返回yield之后表达式的值。

一个简单的例子：使用协程返回值

最简单的协程使用，通过next与send控制协程的运行。

1. def coroutine_print():
2. while True:
3. message = yield
4. print message
6. it = coroutine_print()
7. next(it)
8. it.send('Hello')
9. it.send('World')
11. # result:
12. # Hello
13. # World

稍微复杂一点：向协程发送数据计算平均值

稍微复杂的一个例子，使用yield的返回值。
假设我们希望执行一个流式计算，不断有数据到达，我们希望计算这些数据的平均数。

1. def avg_coroutine():
2. cnt = 0
3. sum = 0
4. new_value = yield
5. while True:
6. sum += new_value
7. cnt += 1
8. new_value = yield float(sum) / cnt

1. In [6]: it = avg_coroutine()
2. In [7]: it.next()
3. In [8]: it.send(10)
4. Out[8]: 10.0
5. In [9]: it.send(20)
6. Out[9]: 15.0
7. In [10]: it.send(30)
8. Out[10]: 20.0
9. In [11]: it.send(110)
10. Out[11]: 42.5

如本例所示，在使用协程时，首先调用next方法停止在第一条yield处，并使用send方法向协程内发送数据，并获取更新后的平均值。

使用throw方法与协程通信

一旦协程开启，仅仅通过send与yield只能对协程进行简单的控制。throw方法提供了在协程内引发异常的接口。通过主叫者调用throw在协程内引发异常，协程捕获异常的方式，可以实现主叫者与协程之间的通信。

需要注意的是，使用throw方法在协程内引发的异常，如果没有被捕获，或者内部又重新raise了不同的异常，那么这个异常会传播到主叫者。

同时throw方法同send与next一样，都会使协程继续运行，并返回下一个yield表达式中的表达式值。且同样的，如果不存在下一个yield表达式协程就结束了，主叫方会收到StopIteration Exception。

使用close方法来结束协程

close方法与throw方法类似，都是在协程暂停的位置引发一个异常，从而向协程发出控制信息。不同于throw方法可以抛出自定义异常，close方法会固定抛出GeneratorExit异常。当这个异常没有被捕获或者引发StopIteration Exception时，close方法会正常返回。这个比较好理解，协程设计者捕获了GeneratorExit异常并完成清理，保证清理干净了，所以最后不会再有yield返回值引发StopItertation。如果因为设计错误导致仍然有下一个yield，那么就会抛出RuntimeError.

Python文档中给出的应用四个协程API的样例。

1. >>> def echo(value=None):
2. ... print "Execution starts when 'next()' is called for the first time."
3. ... try:
4. ... while True:
5. ... try:
6. ... value = (yield value)
7. ... except Exception, e:
8. ... value = e
9. ... finally:
10. ... print "Don't forget to clean up when 'close()' is called."
11. ...
12. >>> generator = echo(1)
13. >>> print generator.next()
14. Execution starts when 'next()' is called for the first time.
15. 1
16. >>> print generator.next()
17. None
18. >>> print generator.send(2)
19. 2
20. >>> generator.throw(TypeError, "spam")
21. TypeError('spam',)
22. >>> generator.close()
23. Don't forget to clean up when 'close()' is called.

这和原来的生成器。这时候使用gen_func.send()会报错，因为没有接受
当某个生成器函数被执行时，生成器函数会开始执行，直到第一条yield表达式为止。

这条表达式会有一个返回值，它取决于使用了生成器的哪个方法去恢复生成器函数的控制流。

next方法会恢复gen func的执行，这里yield表达式会返回值None。
而gen func会继续运行到下一条yield语句，并将yield后面的值反回。

send和next类似，不同的是她可以传入一个值作为yield表达式的计算值。如果用send去开启gen func的执行，那么参数必须为None，因为这时候没有yield语句在等待返回值。

Python文档中的描述

PEP 342 — Coroutines via Enhanced Generators
New in version 2.5.
The yield expression is only used when defining a generator function, and can only be used in the body of a function definition. Using a yield expression in a function definition is sufficient to cause that definition to create a generator function instead of a normal function.

When a generator function is called, it returns an iterator known as a generator. That generator then controls the execution of a generator function. The execution starts when one of the generator’s methods is called. At that time, the execution proceeds to the first yield expression, where it is suspended again, returning the value of expression_list to generator’s caller. By suspended we mean that all local state is retained, including the current bindings of local variables, the instruction pointer, and the internal evaluation stack. When the execution is resumed by calling one of the generator’s methods, the function can proceed exactly as if the yield expression was just another external call. The value of the yield expression after resuming depends on the method which resumed the execution.

All of this makes generator functions quite similar to coroutines; they yield multiple times, they have more than one entry point and their execution can be suspended. The only difference is that a generator function cannot control where should the execution continue after it yields; the control is always transferred to the generator’s caller.
5.2.10.1. Generator-iterator methods
This subsection describes the methods of a generator iterator. They can be used to control the execution of a generator function.
Note that calling any of the generator methods below when the generator is already executing raises a ValueError exception.
generator.next()
Starts the execution of a generator function or resumes it at the last executed yield expression. When a generator function is resumed with a next() method, the current yield expression always evaluates to None. The execution then continues to the next yield expression, where the generator is suspended again, and the value of the expression_list is returned to next()‘s caller. If the generator exits without yielding another value, a StopIteration exception is raised.

1. generator.send(value)

Resumes the execution and “sends” a value into the generator function. The value argument becomes the result of the current yield expression. The send() method returns the next value yielded by the generator, or raises StopIteration if the generator exits without yielding another value. When send() is called to start the generator, it must be called with Noneas the argument, because there is no yield expression that could receive the value.

1. generator.throw(type[, value[, traceback]])

Raises an exception of type type at the point where generator was paused, and returns the next value yielded by the generator function. If the generator exits without yielding another value, a StopIteration exception is raised. If the generator function does not catch the passed-in exception, or raises a different exception, then that exception propagates to the caller.

1. generator.close()

Raises a GeneratorExit at the point where the generator function was paused. If the generator function then raises StopIteration (by exiting normally, or due to already being closed) or GeneratorExit (by not catching the exception), close returns to its caller. If the generator yields a value, a RuntimeError is raised. If the generator raises any other exception, it is propagated to the caller. close() does nothing if the generator has already exited due to an exception or normal exit.

Here is a simple example that demonstrates the behavior of generators and generator functions:

1. >>> def echo(value=None):
2. ... print "Execution starts when 'next()' is called for the first time."
3. ... try:
4. ... while True:
5. ... try:
6. ... value = (yield value)
7. ... except Exception, e:
8. ... value = e
9. ... finally:
10. ... print "Don't forget to clean up when 'close()' is called."
11. ...
12. >>> generator = echo(1)
13. >>> print generator.next()
14. # Execution starts when 'next()' is called for the first time.
15. 1
16. >>> print generator.next()
17. None
18. >>> print generator.send(2)
19. 2
20. >>> generator.throw(TypeError, "spam")
21. TypeError('spam',)
22. >>> generator.close()
23. # Don't forget to clean up when 'close()' is called.