Friday, November 6, 2026

Advanced Functions and OOP

1. Closures in Python

1.1 What is a Closure?

A closure is a function that remembers variables from its enclosing scope, even after the outer function has finished executing.

Closures are useful for:

  • Maintaining state without global variables.

  • Creating configurable functions.

  • Implementing decorators.

  • Building function factories.

1.2 Understanding Closures

Consider this example:

def outer_function(message):

    def inner_function():
        print(message)

    return inner_function


my_function = outer_function("Hello Python!")

my_function()

Output:

Hello Python!

Explanation:

  1. outer_function() receives a message.

  2. It defines an inner function.

  3. The inner function uses the outer function's variable.

  4. The outer function returns the inner function.

  5. Even after outer_function() finishes, the inner function remembers message.

That is a closure.

1.3 Closure with State

def counter():

    count = 0

    def increment():

        nonlocal count
        count += 1

        return count

    return increment


my_counter = counter()

print(my_counter())
print(my_counter())
print(my_counter())

Output:

1
2
3

Important concept: nonlocal allows an inner function to modify a variable defined in its enclosing function.

Without nonlocal, Python treats an assignment to count as a new local variable.

1.4 Practical Example: Bank Account

def create_account(initial_balance):

    balance = initial_balance

    def deposit(amount):
        nonlocal balance
        balance += amount
        return balance

    def withdraw(amount):
        nonlocal balance

        if amount <= balance:
            balance -= amount
        else:
            print("Insufficient balance")

        return balance

    return deposit, withdraw


deposit, withdraw = create_account(1000)

print(deposit(500))
print(withdraw(200))

Output:

1500
1300

Interview question: What is the difference between a closure and a global variable?

Answer: A closure maintains access to variables in its enclosing lexical scope. A global variable belongs to the module-level namespace and can be accessed by functions throughout that module.


2. Decorators in Python

2.1 What is a Decorator?

A decorator is a function that modifies or extends the behavior of another function without changing its original source code.

Decorators are commonly used for:

  • Logging

  • Authentication

  • Authorization

  • Performance monitoring

  • Caching

  • Retry mechanisms

  • Input validation

2.2 Basic Decorator

def my_decorator(func):

    def wrapper():
        print("Before function execution")

        func()

        print("After function execution")

    return wrapper


@my_decorator
def say_hello():
    print("Hello World!")


say_hello()

Output:

Before function execution
Hello World!
After function execution

The following:

@my_decorator
def say_hello():
    print("Hello World!")

is equivalent to:

def say_hello():
    print("Hello World!")

say_hello = my_decorator(say_hello)

2.3 Decorators with Arguments

A decorator should support functions that accept different parameters.

def my_decorator(func):

    def wrapper(*args, **kwargs):

        print("Function started")

        result = func(*args, **kwargs)

        print("Function completed")

        return result

    return wrapper


@my_decorator
def add(a, b):
    return a + b


print(add(10, 20))

Output:

Function started
Function completed
30

Important:

  • *args accepts positional arguments.

  • **kwargs accepts keyword arguments.

  • return result preserves the original function's return value.

2.4 Decorators with Their Own Arguments

Sometimes we want to configure the decorator itself.

Example: Repeat a function a specified number of times.

def repeat(times):

    def decorator(func):

        def wrapper(*args, **kwargs):

            for _ in range(times):
                result = func(*args, **kwargs)

            return result

        return wrapper

    return decorator


@repeat(3)
def greet():
    print("Hello!")


greet()

Output:

Hello!
Hello!
Hello!

Notice the three levels:

  1. repeat(times) receives decorator configuration.

  2. decorator(func) receives the original function.

  3. wrapper(*args, **kwargs) executes the function.

2.5 Preserve Function Metadata Using functools.wraps

Decorators can overwrite the original function's name and documentation.

import functools

def my_decorator(func):

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)

    return wrapper


@my_decorator
def calculate():
    """Calculate something."""
    return 100


print(calculate.__name__)
print(calculate.__doc__)

Output:

calculate
Calculate something.

Always consider using functools.wraps when writing decorators.

2.6 Class-Based Decorators

Python also allows us to implement decorators using classes.

A class-based decorator implements the __call__() method.

import functools

class Logger:

    def __init__(self, func):
        functools.update_wrapper(self, func)
        self.func = func

    def __call__(self, *args, **kwargs):

        print(f"Calling {self.func.__name__}")

        result = self.func(*args, **kwargs)

        print("Execution completed")

        return result


@Logger
def add(a, b):
    return a + b


print(add(10, 20))

Output:

Calling add
Execution completed
30

The class instance behaves like a function because it implements __call__().

2.7 Class-Based Decorator with Arguments

import functools

class Repeat:

    def __init__(self, times):
        self.times = times

    def __call__(self, func):

        @functools.wraps(func)
        def wrapper(*args, **kwargs):

            result = None

            for _ in range(self.times):
                result = func(*args, **kwargs)

            return result

        return wrapper


@Repeat(3)
def greet():
    print("Welcome!")


greet()

This approach is useful when a decorator requires configuration or maintains internal state.


3. The functools Module

The functools module provides tools for working with higher-order functions and callable objects.

3.1 functools.wraps()

Preserves the metadata of the original function.

from functools import wraps

Use it when creating custom decorators.

3.2 functools.lru_cache()

Caches the result of function calls.

from functools import lru_cache

@lru_cache(maxsize=128)
def fibonacci(n):

    if n < 2:
        return n

    return fibonacci(n - 1) + fibonacci(n - 2)


print(fibonacci(10))

print(fibonacci.cache_info())

Output:

55
CacheInfo(hits=8, misses=11, maxsize=128, currsize=11)

Benefits:

  • Reduces repeated calculations.

  • Improves performance.

  • Useful for recursive functions.

The arguments must be hashable, and cached results should be safe to reuse.

3.3 functools.cache()

An unbounded caching decorator.

from functools import cache

@cache
def square(n):
    print("Calculating...")
    return n * n


print(square(5))
print(square(5))

The calculation runs only once for the same argument.

3.4 functools.partial()

Creates a new function by fixing some arguments of an existing function.

from functools import partial

def multiply(a, b):
    return a * b


double = partial(multiply, 2)

print(double(10))
print(double(20))

Output:

20
40

3.5 functools.singledispatch()

Allows a function to have different implementations based on the type of its first argument.

from functools import singledispatch

@singledispatch
def display(value):
    print("Default:", value)


@display.register
def _(value: int):
    print("Integer:", value)


@display.register
def _(value: list):
    print("List:", value)


display(100)
display([1, 2, 3])
display("Python")

This is useful when implementing type-specific behavior.


4. Context Managers and contextlib

4.1 What is a Context Manager?

A context manager manages resources and ensures that cleanup operations happen correctly.

Common examples:

  • Opening and closing files.

  • Managing database connections.

  • Managing locks.

  • Handling transactions.

  • Managing temporary resources.

Python provides the with statement for context managers.

4.2 File Handling Example

with open("sample.txt", "w") as file:
    file.write("Learning Python")

The file is automatically closed when the with block exits, including when an exception occurs.

4.3 Creating a Custom Context Manager

A context manager class implements:

  • __enter__()

  • __exit__()

class MyContext:

    def __enter__(self):
        print("Entering context")
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        print("Exiting context")


with MyContext():
    print("Inside context")

Output:

Entering context
Inside context
Exiting context

The __exit__() method receives exception information.

Returning True from __exit__() suppresses the exception. Returning False or None allows it to propagate.

4.4 Using contextlib.contextmanager

The contextlib module provides a simpler way to create context managers using generators.

from contextlib import contextmanager

@contextmanager
def my_context():

    print("Starting")

    try:
        yield
    finally:
        print("Cleaning up")


with my_context():
    print("Executing task")

Output:

Starting
Executing task
Cleaning up

The yield statement separates setup from the managed block and cleanup.

4.5 Other Useful contextlib Features

closing(): Ensures that an object with a close() method is closed.

from contextlib import closing

with closing(open("sample.txt")) as file:
    print(file.read())

suppress(): Suppresses specified exceptions.

from contextlib import suppress

with suppress(FileNotFoundError):
    open("missing.txt")

ExitStack: Manages multiple context managers dynamically.

from contextlib import ExitStack

with ExitStack() as stack:

    files = [
        stack.enter_context(open("a.txt")),
        stack.enter_context(open("b.txt"))
    ]

    print("Both files are open")

5. Descriptors in Python

5.1 What is a Descriptor?

A descriptor is an object that defines one or more of these methods:

  • __get__()

  • __set__()

  • __delete__()

Descriptors control how attributes are accessed, assigned, and deleted.

They are used internally by properties, methods, and many Python frameworks.

5.2 Basic Descriptor Example

class Age:

    def __get__(self, instance, owner):
        return instance._age

    def __set__(self, instance, value):

        if value < 0:
            raise ValueError("Age cannot be negative")

        instance._age = value


class Person:

    age = Age()


person = Person()

person.age = 25

print(person.age)

Output:

25

Trying to assign a negative age raises a ValueError.

5.3 Data and Non-Data Descriptors

A data descriptor implements __set__() or __delete__().

A non-data descriptor implements only __get__().

Data descriptors generally take precedence over an instance's dictionary when resolving attributes.

5.4 Practical Uses

Descriptors are useful for:

  • Attribute validation.

  • Type checking.

  • Lazy loading.

  • Reusable properties.

  • ORM field management.

  • Framework-level attribute management.


6. Metaclasses in Python

6.1 What is a Metaclass?

A metaclass is a class whose instances are classes.

In Python, the default metaclass is type.

class Employee:
    pass


print(type(Employee))

Output:

<class 'type'>

Think of it this way:

  • Objects are created from classes.

  • Classes are created by metaclasses.

6.2 Creating a Custom Metaclass

class MyMeta(type):

    def __new__(cls, name, bases, namespace):

        print(f"Creating class: {name}")

        return super().__new__(
            cls, name, bases, namespace
        )


class Employee(metaclass=MyMeta):
    pass

Output:

Creating class: Employee

The __new__() method participates in class creation.

6.3 Practical Example: Automatically Register Classes

class PluginMeta(type):

    registry = {}

    def __new__(cls, name, bases, namespace):

        new_class = super().__new__(
            cls, name, bases, namespace
        )

        if name != "Plugin":
            cls.registry[name] = new_class

        return new_class


class Plugin(metaclass=PluginMeta):
    pass


class EmailPlugin(Plugin):
    pass


class PaymentPlugin(Plugin):
    pass


print(PluginMeta.registry)

This pattern is useful for plugin architectures and framework registration.

Important: Metaclasses are powerful but should be used only when simpler alternatives are insufficient.


7. init_subclass()

__init_subclass__() is called whenever a new subclass is created.

It provides a simpler alternative to many metaclass use cases.

Example

class Plugin:

    registry = []

    def __init_subclass__(cls, **kwargs):

        super().__init_subclass__(**kwargs)

        Plugin.registry.append(cls)


class EmailPlugin(Plugin):
    pass


class PaymentPlugin(Plugin):
    pass


print(Plugin.registry)

This automatically registers subclasses.

When should you use it?

Use __init_subclass__() when you want to:

  • Validate subclasses.

  • Register implementations.

  • Enforce class conventions.

  • Initialize subclass-level configuration.

Prefer it over a metaclass when you only need to customize subclass creation.


8. Dataclasses

8.1 What is a Dataclass?

A dataclass simplifies the creation of classes primarily used to store data.

The @dataclass decorator automatically generates methods such as:

  • __init__()

  • __repr__()

  • __eq__()

Example Without Dataclass

class Employee:

    def __init__(self, name, salary):
        self.name = name
        self.salary = salary

Same Example Using Dataclass

from dataclasses import dataclass

@dataclass
class Employee:

    name: str
    salary: float


emp = Employee("Rahul", 50000)

print(emp)

Output:

Employee(name='Rahul', salary=50000)

8.2 Default Values

from dataclasses import dataclass

@dataclass
class Product:

    name: str
    price: float
    quantity: int = 1


product = Product("Laptop", 50000)

print(product)

8.3 default_factory

Use default_factory for mutable default values.

from dataclasses import dataclass, field

@dataclass
class Student:

    name: str
    subjects: list = field(default_factory=list)


s1 = Student("Amit")
s2 = Student("Neha")

s1.subjects.append("Python")

print(s1.subjects)
print(s2.subjects)

Output:

['Python']
[]

Each object receives its own list.

8.4 Frozen Dataclasses

from dataclasses import dataclass

@dataclass(frozen=True)
class Configuration:

    host: str
    port: int


config = Configuration("localhost", 8000)

A frozen dataclass prevents normal attribute reassignment after initialization.


9. Protocols and Abstract Base Classes

These concepts help build flexible and maintainable application architectures.

9.1 Abstract Base Classes (ABCs)

An ABC defines a contract that subclasses must implement.

from abc import ABC, abstractmethod

class PaymentProcessor(ABC):

    @abstractmethod
    def process_payment(self, amount):
        pass


class CreditCardPayment(PaymentProcessor):

    def process_payment(self, amount):
        print(f"Processing payment: {amount}")


payment = CreditCardPayment()

payment.process_payment(1000)

A subclass that does not implement all abstract methods cannot be instantiated.

9.2 Why Use ABC?

Use ABC when:

  • You want explicit inheritance.

  • You want to enforce implementation requirements.

  • You need shared behavior.

  • You are designing a framework or application architecture.

9.3 Protocol

A Protocol supports structural subtyping.

This means a class can satisfy an interface by implementing the required methods, without explicitly inheriting from that interface.

from typing import Protocol

class PaymentProcessor(Protocol):

    def process_payment(self, amount: float) -> None:
        ...


class CreditCardPayment:

    def process_payment(self, amount: float) -> None:
        print(f"Paid {amount}")


def checkout(
    processor: PaymentProcessor,
    amount: float
):
    processor.process_payment(amount)


checkout(CreditCardPayment(), 500)

The CreditCardPayment class does not need to inherit from PaymentProcessor.

9.4 ABC vs Protocol

FeatureABCProtocol
Typing approachNominalStructural
Explicit inheritanceUsually requiredNot required
Abstract methodsYesMethod signatures
Runtime enforcementYes, for abstract methodsNot by default
Main purposeEnforce contractsFlexible interfaces

Use ABC when inheritance and runtime enforcement are important.

Use Protocol when you want flexible interfaces and better static type checking.


10. LAB PROJECT: Build a Retry, Caching and Timing Decorator Library

Project Objective

Build a reusable Python library that provides three decorators:

  1. @retry – Automatically retries failed function calls.

  2. @cache_result – Caches function results.

  3. @timing – Measures function execution time.

These decorators should work with ordinary functions and support function arguments.

Project Structure

week2_project/
│
├── decorators.py
├── main.py
└── requirements.txt

We will use Python's standard library, so no external packages are required.

Step 1: Implement Timing Decorator

Create decorators.py.

import functools
import time


def timing(func):

    @functools.wraps(func)
    def wrapper(*args, **kwargs):

        start = time.perf_counter()

        try:
            return func(*args, **kwargs)

        finally:
            elapsed = time.perf_counter() - start

            print(
                f"{func.__name__} executed in "
                f"{elapsed:.4f} seconds"
            )

    return wrapper

Explanation:

  • time.perf_counter() measures elapsed time.

  • functools.wraps() preserves function metadata.

  • finally ensures timing is printed even if the function raises an exception.

Step 2: Implement Retry Decorator

Add the following code to decorators.py.

def retry(max_attempts=3, delay=1, exceptions=(Exception,)):

    def decorator(func):

        @functools.wraps(func)
        def wrapper(*args, **kwargs):

            for attempt in range(1, max_attempts + 1):

                try:
                    return func(*args, **kwargs)

                except exceptions:

                    if attempt == max_attempts:
                        raise

                    print(
                        f"Attempt {attempt} failed. "
                        f"Retrying..."
                    )

                    time.sleep(delay)

        return wrapper

    return decorator

Usage:

@retry(max_attempts=3, delay=2)
def connect_api():

    print("Connecting to API")

    raise ConnectionError("Server unavailable")


connect_api()

The function will execute three times before the final exception is raised.

Important production considerations:

  • Retry only transient failures.

  • Use exponential backoff for real services.

  • Avoid retrying non-idempotent operations unless they are designed to be safe to retry.

  • Consider adding jitter and maximum delay.

Step 3: Implement Caching Decorator

Add this code to decorators.py.

def cache_result(maxsize=128):

    def decorator(func):

        cached_func = functools.lru_cache(
            maxsize=maxsize
        )(func)

        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            return cached_func(*args, **kwargs)

        wrapper.cache_info = cached_func.cache_info
        wrapper.cache_clear = cached_func.cache_clear

        return wrapper

    return decorator

Usage:

@cache_result(maxsize=100)
def calculate_square(number):

    print("Calculating...")

    return number * number


print(calculate_square(10))
print(calculate_square(10))

print(calculate_square.cache_info())

The second call reuses the cached result.

Note: The cache uses function arguments as keys. Arguments must be hashable. Cached functions should generally avoid side effects, and cache invalidation should be considered when underlying data changes.

Step 4: Use All Three Decorators Together

Create main.py.

from decorators import timing, retry, cache_result

@timing
@retry(max_attempts=3, delay=1)
@cache_result(maxsize=100)
def fetch_data(user_id):

    print(f"Fetching data for user {user_id}")

    return {
        "user_id": user_id,
        "name": "Rahul"
    }


print(fetch_data(101))

print(fetch_data(101))

The decorators are applied from bottom to top:

  1. cache_result wraps the original function.

  2. retry wraps the cached function.

  3. timing wraps the retry-enabled function.

The execution flow is important because decorator order changes behavior.

Step 5: Test Your Library

Test the following scenarios:

  1. A function that succeeds on the first attempt.

  2. A function that fails twice and succeeds on the third attempt.

  3. A function that always fails.

  4. A function that returns the same result for repeated arguments.

  5. A function that receives keyword arguments.

  6. A function that takes several seconds to execute.

  7. Two different functions using the same decorators.

Suggested testing framework: pytest.


11. Additional Advanced Challenges

Once the main project is complete, try these extensions:

Challenge 1: Exponential Backoff

Instead of a fixed delay, implement:

delay = initial_delay * (2 ** (attempt - 1))

Challenge 2: Cache Expiration

Add a TTL (Time To Live) option so cached results expire after a specified number of seconds.

Challenge 3: Class-Based Timing Decorator

Implement timing using a class with __init__() and __call__().

Challenge 4: Logging Decorator

Create a decorator that logs:

  • Function name

  • Input arguments

  • Execution time

  • Return value

  • Exceptions

Avoid logging passwords, tokens, or other sensitive arguments.

Challenge 5: Async Retry Decorator

Extend the retry decorator to support async def functions using asyncio.sleep().


12. Week 2 Interview Questions

  1. What is a closure in Python?

  2. What is the purpose of nonlocal?

  3. What is a decorator?

  4. What is the difference between a decorator and a closure?

  5. What is the purpose of functools.wraps()?

  6. How do decorators with arguments work?

  7. How do class-based decorators work?

  8. What is the difference between cache and lru_cache?

  9. What is functools.partial()?

  10. What is a context manager?

  11. What are __enter__() and __exit__()?

  12. What is the purpose of contextlib?

  13. What is a descriptor?

  14. What is the difference between data and non-data descriptors?

  15. What is a metaclass?

  16. What is the default metaclass in Python?

  17. What is the purpose of __init_subclass__()?

  18. What is a dataclass?

  19. What is the purpose of default_factory?

  20. What is a frozen dataclass?

  21. What is an Abstract Base Class?

  22. What is a Protocol?

  23. What is the difference between ABC and Protocol?

  24. What happens when multiple decorators are applied to a function?

  25. How would you implement a retry decorator for an API?


13. Week 2 Completion Checklist

  • Understand closures and lexical scoping.

  • Implement decorators with and without arguments.

  • Implement class-based decorators.

  • Use functools.wraps, lru_cache, cache, and partial.

  • Create custom context managers.

  • Understand descriptors.

  • Understand metaclasses and __init_subclass__.

  • Implement dataclasses.

  • Understand ABCs and Protocols.
  • Complete the retry decorator.

  • Complete the caching decorator.

  • Complete the timing decorator.

  • Test the library using pytest.

Final Outcome

At the end of Week 2, you should be able to design reusable Python components and understand many of the advanced mechanisms used internally by professional Python frameworks and libraries.

Recommended hands-on study sequence

Day

Topic

Practical assignment

Day 1

Closures

Build a counter and bank account

Day 2

Decorators

Logging and authentication decorators

Day 3

functools

Fibonacci caching and partial functions

Day 4

Context managers

Database connection manager

Day 5

Descriptors and metaclasses

Attribute validation and plugin registry

Day 6

Dataclasses, ABC, Protocol

Payment processing architecture

Day 7

Lab project

Build and test the decorator library


No comments:

Post a Comment

Advanced Functions and Object-Oriented Python

Level: Intermediate to advanced Suggested time: 5–7 hours Python: 3.10+ (examples use standard-library features) This week moves beyond w...