2.4. Generator Function

2.4.1. Rationale

  • yield keyword turns function into generator function

2.4.2. Recap

>>> def run():
...     return 1
>>>
>>>
>>> result = run()
>>> print(result)
1
>>> def run():
...     return 1
...     return 2  # this will not execute
>>>
>>>
>>> result = run()
>>> print(result)
1
>>> def run():
...     return 1, 2
>>>
>>>
>>> result = run()
>>> print(result)
(1, 2)

2.4.3. Definition

Generators can return (or yield) something:

>>> def run():
...     yield 'something'

Generators can be defined with required and optional parameters just like a regular function:

>>> def mygenerator(a, b, c=0):
...     yield a + b + c

2.4.4. Call Generator

Generators are called just like a regular function:

>>> def mygenerator():
...     yield 'something'
>>>
>>>
>>> result = mygenerator()

The rule with positional and keyword arguments are identical tp regular functions:

>>> def mygenerator(a, b, c=0):
...     yield a + b + c
>>>
>>>
>>> result = mygenerator(1, b=2)

2.4.5. Get Results

  • All generators implements Iterator protocol

  • Iterator has obj.__iter__() method which enable use of iter(obj)

  • Iterator has obj.__next__() method which enable use of next(obj)

One does not simple get data from generator:

>>> def run():
...     yield 1
>>>
>>>
>>> result = run()
>>> print(result)  
<generator object run at 0x...>

In Order to do so, you need to generate next item using next()

>>> def run():
...     yield 1
>>>
>>>
>>> result = run()
>>> next(result)
1
>>> next(result)
Traceback (most recent call last):
StopIteration

2.4.6. Yield Keyword

>>> def run():
...     yield 1
>>>
>>>
>>> result = run()
>>>
>>> next(result)
1
>>> next(result)
Traceback (most recent call last):
StopIteration
>>> def run():
...     yield 1
...     yield 2
...     yield 3
>>>
>>>
>>> result = run()
>>>
>>> next(result)
1
>>> next(result)
2
>>> next(result)
3
>>> next(result)
Traceback (most recent call last):
StopIteration
>>> def run():
...     print('a')
...     print('aa')
...     yield 1
...     print('b')
...     print('bb')
...     yield 2
...     print('c')
...     print('cc')
...     yield 3
>>>
>>>
>>> result = run()
>>>
>>> next(result)
a
aa
1
>>> next(result)
b
bb
2
>>> next(result)
c
cc
3
>>> next(result)
Traceback (most recent call last):
StopIteration

2.4.7. Yield in a Loop

>>> def run():
...     for x in range(0,3):
...         yield x
>>>
>>>
>>> result = run()
>>>
>>> next(result)
0
>>> next(result)
1
>>> next(result)
2
>>> next(result)
Traceback (most recent call last):
StopIteration

2.4.8. Yields in Loops

>>> def run():
...     for x in range(0, 3):
...         yield x
...     for y in range(10, 13):
...         yield y
>>>
>>>
>>> result = run()
>>>
>>> type(result)
<class 'generator'>
>>>
>>> next(result)
0
>>> next(result)
1
>>> next(result)
2
>>> next(result)
10
>>> next(result)
11
>>> next(result)
12
>>> next(result)
Traceback (most recent call last):
StopIteration

2.4.9. Yield in a Zip Loop

>>> def names():
...     yield 'Mark Watney'
...     yield 'Melissa Lewis'
...     yield 'Rick Martinez'
>>>
>>>
>>> def roles():
...     yield 'botanist'
...     yield 'commander'
...     yield 'pilot'
>>>
>>>
>>> for n, r in zip(names(), roles()):
...     print(r, n)
botanist Mark Watney
commander Melissa Lewis
pilot Rick Martinez

2.4.10. Example

Function:

>>> def even(data):
...     result = []
...     for x in data:
...         if x % 2 == 0:
...             result.append(x)
...     return result
>>>
>>>
>>> DATA = [0, 1, 2, 3, 4, 5]
>>>
>>> result = even(DATA)
>>>
>>> print(result)
[0, 2, 4]

Generator:

>>> def even(data):
...     for x in data:
...         if x % 2 == 0:
...             yield x
>>>
>>>
>>> DATA = [0, 1, 2, 3, 4, 5]
>>>
>>> result = even(DATA)
>>>
>>> print(result)  
<generator object even at 0x...>
>>> list(result)
[0, 2, 4]

2.4.11. Assignments

Code 2.27. Solution
"""
* Assignment: Generator Function Passwd
* Complexity: medium
* Lines of code: 10 lines
* Time: 8 min

English:
    1. Split `DATA` by lines and then by colon `:`
    2. Extract system accounts (users with UID [third field] is less than 1000)
    3. Return list of system account logins
    4. Implement solution using function
    5. Implement solution using generator and `yield` keyword
    6. Compare results of both using `sys.getsizeof()`
    7. Run doctests - all must succeed

Polish:
    1. Podziel `DATA` po liniach a następnie po dwukropku `:`
    2. Wyciągnij konta systemowe (użytkownicy z UID (trzecie pole) mniejszym niż 1000)
    3. Zwróć listę loginów użytkowników systemowych
    4. Zaimplementuj rozwiązanie wykorzystując funkcję
    5. Zaimplementuj rozwiązanie wykorzystując generator i słowo kluczowe `yield`
    6. Porównaj wyniki obu używając `sys.getsizeof()`
    7. Uruchom doctesty - wszystkie muszą się powieść

Hints:
    * ``str.splitlines()``
    * ``str.split()``
    * unpacking expression

Tests:
    >>> import sys; sys.tracebacklimit = 0
    >>> from sys import getsizeof
    >>> from inspect import isfunction, isgeneratorfunction

    >>> assert isfunction(function)
    >>> assert isgeneratorfunction(generator)

    >>> list(function(DATA))
    ['root', 'bin', 'daemon', 'adm', 'shutdown', 'halt', 'nobody', 'sshd']

    >>> list(generator(DATA))
    ['root', 'bin', 'daemon', 'adm', 'shutdown', 'halt', 'nobody', 'sshd']
"""

DATA = """root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
nobody:x:99:99:Nobody:/:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
watney:x:1000:1000:Mark Watney:/home/watney:/bin/bash
jimenez:x:1001:1001:José Jiménez:/home/jimenez:/bin/bash
ivanovic:x:1002:1002:Иван Иванович:/home/ivanovic:/bin/bash
lewis:x:1003:1002:Melissa Lewis:/home/ivanovic:/bin/bash"""


# Callable: list[str] with usernames when UID [third field] is less than 1000
def function(data: str):
    ...


# Generator: list[str] with usernames when UID [third field] is less than 1000
def generator(data: str):
    ...