2. Generators
• Generators simplifies creation of iterators.
• A generator is a function that produces a sequence of results
instead of a single value.
• A generator is a special routine that can be used to control the
iteration behaviour of a loop.
• A generator is similar to a function returning an array.
• A generator has parameters, it can be called and it generates a
sequence of result.
• But unlike functions, which return a whole array, a
generator yields one value at a time.
3. Generators
Generators in Python:
• Are defined with the def keyword
• Use the yield keyword
• May use several yield keywords
• Return an iterator
4. Examples
def yrange(n):
i = 0
while i < n:
yield i
i += 1
Each time the yield statement is executed the function
generates a new value.
5. Examples
>>> y = yrange(3)
>>> y
<generator object yrange at 0x401f30>
>>> y.next()
0
>>> y.next()
1
>>> y.next()
2
>>> y.next()
Traceback (most recent call last):
File <stdin>, line 1, in <module>
StopIteration
6. Examples
• A generator is also an iterator.
• The word “generator” is confusingly used to mean both the
function that generates and what it generates.
• When a generator function is called, it returns a generator
object without even beginning execution of the function.
• When next method is called for the first time, the function
starts executing until it reaches yield statement.
• The yielded value is returned by the next call.
7. Examples
>>> def foo():
... print "begin"
... for i in range(3):
... print "before yield", i
... yield i
... print "after yield", i
... print "end"
... >>> f = foo()
>>> f.next()
begin before yield 0
0
>>> f.next()
after yield 0
before yield 1
1
>>> f.next()
after yield 1
before yield 2
2
>>> f.next()
after yield 2
end
Traceback (most recent call last):
File <stdin>, line 1, in <module>
StopIteration
>>>
8. Examples
• Lets say we want to write a program that takes a list of filenames as
arguments and prints contents of all those files.
• The traditional way to implement it is:
def cat(filenames):
for f in filenames:
for line in open(f):
print line,
9. Examples
• Now, lets say we want to print only the line which has a
particular substring.
def grep(pattern, filenames):
for f in filenames:
for line in open(f):
if pattern in line:
print line,
• Both these programs have lot of code in common. It is hard to
move the common part to a function. But with generators
makes it possible to do it.
10. Examples
def readfiles(filenames):
for f in filenames:
for line in open(f):
yield line
def grep(pattern, lines):
return (line for line in lines if
pattern in line)
def printlines(lines):
for line in lines:
print line,
def main(pattern, filenames):
lines = readfiles(filenames)
lines = grep(pattern, lines)
printlines(lines)
11. Generator expressions
• There are two types of generators in Python:
generator functions and generator expressions.
• A generator function is any function in which the keyword yield
appears in its body.
• The appearance of the keyword yield is enough to make the
function a generator function.
• The other type of generators are the generator equivalent of a
list comprehension. Its syntax is really elegant for a limited use
case.
12. Examples
>>> numbers = [1, 2, 3, 4, 5, 6]
>>> [x * x for x in numbers]
[1, 4, 9, 16, 25, 36]
You could do the same thing with a set comprehension:
>>> {x * x for x in numbers}
{1, 4, 36, 9, 16, 25}
Or a dict comprehension:
>>> {x: x * x for x in numbers}
{1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36}
14. Advantages
• Cleaner code
• Iterators can work with infinite sequences
• Iterators save resources
• Generator allows to write streaming code with fewer
intermediate variables and data structures.
• Memory and CPU efficient