Erik Groeneveld: Generators
Erik Groeneveld talks about generators, at PyGrunn.
See the PyGrunn website for more info about this one-day Python conference in Groningen, The Netherlands.
I started my own company, Seecr. This talk is about PEP 342 (Coroutines via Enhanced Generators) and PEP 380 (Syntax for Delegating to a Subgenerator) and beyond. Most of this session is detailed on http://weightless.io/compose
I will talk about five simple things. Once you start working with it, it can get quite complicated, but it is really great.
Basis
Basis: list comprehensions with round instead of square brackets. It gives you a lazy thing instead of a completely populated list. You call .next() on the result to get the next item. Very useful for creating lazy and efficient programs.
Generator functions: using yield instead of return will turn a function automatically into a generator.
Generalized generators, PEP 342
Accept input for a generator: use .send(value) to send a new value to a generator. The generator then goes both ways, and then you basically have a coroutine.
Decomposition, PEP 380
Decomposing a function into two functions is easy. With generators it was not possible. Now it is. Add the @compose decorator to the main generator and call yield sub() on the sub generator. On Python 3.3 and higher, it is nicer: yield from sub(). But you need two new concepts to write real programs with it.
Push back, beyond
Implicitly at the end of a normal function, there is return None if there is no explicit return. For generator functions, you implicitly get raise StopIteration. With Python 3.3 and higher you can explicitly return 42 which translates into raise StopIteration(42) to return a value.
But you can also then do raise StopIteration(42, 56, 18). We will use this to push data back into the stream. The main generator will get 42, and then the next main yield will give you 56, and then 18.
None protocol, beyond
If you yield nothing, you return None. So yield None is the same. With yield None you get the generator ready to receive data, putting it at the first yield statement, so you do not need to first call .next() on it. You alternate the reading and writing stage. You send a few values, and then you tell the generator that you are ready to receive data again by yielding None to it.
Applications
Weightless is an I/O framework. It ties generators to sockets. See http://weightless.io/weightless and http://weightless.io/compose and source code at https://github.com/seecr/weightless-core