Returning Functions ​

Higher-order functions can not only accept functions as arguments, but also return functions as results.

Let's implement a summation with variable parameters. Normally, a summation function is defined like this:

def calc_sum(*args):
    ax = 0
    for n in args:
        ax = ax + n
    return ax

However, what if we don't need to calculate the sum immediately and would rather compute it later as needed? In that case, we can return a summation function instead of the result:

def lazy_sum(*args):
    def sum():
        ax = 0
        for n in args:
            ax = ax + n
        return ax
    return sum

When we call lazy_sum(), it returns a summation function rather than the sum itself:

>>> f = lazy_sum(1, 3, 5, 7, 9)
>>> f
<function lazy_sum.<locals>.sum at 0x101c6ed90>

The actual sum is calculated only when the function f is called:

>>> f()
25

In this example, we defined a function sum inside the function lazy_sum, and the internal function sum can reference the parameters and local variables of lazy_sum. When lazy_sum returns the function sum, the parameters and variables are preserved in the returned function, creating a powerful structure called a "closure."

Take note that each call to lazy_sum() returns a new function, even if the arguments are identical:

>>> f1 = lazy_sum(1, 3, 5, 7, 9)
>>> f2 = lazy_sum(1, 3, 5, 7, 9)
>>> f1 == f2
False

The results of calling f1() and f2() do not interfere with each other.

Closures ​

Notice that the returned function references local variables of its enclosing function, meaning those local variables are retained even after the enclosing function returns. Therefore, closures are easy to use but not so easy to implement.

Another important point is that the returned function does not execute immediately; it only executes when it is called. Let's look at an example:

def count():
    fs = []
    for i in range(1, 4):
        def f():
             return i * i
        fs.append(f)
    return fs

f1, f2, f3 = count()

In this example, a new function is created with each iteration, and all three functions are returned.

You might think that calling f1(), f2(), and f3() would result in 1, 4, and 9, respectively. But the actual results are:

>>> f1()
9
>>> f2()
9
>>> f3()
9

All results are 9! This is because the returned functions reference the variable i, but they do not execute immediately. By the time all three functions are returned, the variable i has changed to 3, making the final result 9.

To reference a loop variable, you can create another function that uses the variable as a parameter, thus binding the current value to the function parameter. This prevents changes to the loop variable from affecting the function:

def count():
    def f(j):
        def g():
            return j * j
        return g
    fs = []
    for i in range(1, 4):
        fs.append(f(i)) # f(i) is executed immediately, passing the current value of i
    return fs

Now let's see the results:

>>> f1, f2, f3 = count()
>>> f1()
1
>>> f2()
4
>>> f3()
9

The downside is that the code is longer, but using a lambda function can make it more concise.

nonlocal ​

Using closures involves the inner function referencing the outer function's local variables. If we only read the outer variable's value, the returned closure works correctly:

def inc():
    x = 0
    def fn():
        # Only reading x's value:
        return x + 1
    return fn

f = inc()
print(f()) # 1
print(f()) # 1

However, if we attempt to modify the outer variable, the Python interpreter will treat x as a local variable of the function fn() and throw an error:

def inc():
    x = 0
    def fn():
        # nonlocal x
        x = x + 1
        return x
    return fn

f = inc()
print(f()) # 1
print(f()) # 2

The reason is that x is treated as an uninitialized local variable. We want to reference the x defined in inc(), so we need to add a nonlocal x statement in fn(). This tells the interpreter that x belongs to the enclosing function's scope, allowing correct calculations.

Exercise ​

Create a counter function using closures, returning an incrementing integer with each call:

def createCounter():
    def counter():
        return 1
    return counter

# Testing:
counterA = createCounter()
print(counterA(), counterA(), counterA(), counterA(), counterA()) # 1 2 3 4 5
counterB = createCounter()
if [counterB(), counterB(), counterB(), counterB()] == [1, 2, 3, 4]:
    print('Test passed!')
else:
    print('Test failed!')

Summary ​

A function can return either a calculation result or another function.

When returning a function, remember that the returned function has not been executed, and avoid referencing variables that may change.