Object-Oriented Programming Greatest Hits

Introduction

#

A programming paradigm is a way of organizing programs. Imperative code specifies how a result is computed, as an explicit sequence of steps that read and modify state. This is usually what people think of when they hear “programming,” since it’s the closest match to algorithmic thinking. Declarative code specifies what the result is or should look like, and an engine derives the steps in the background. AI-assisted programming has also shifted toward a more declarative interaction model. Developers increasingly describe the behavior they want while models derive implementation steps.

Within imperative programming, procedural code organizes the steps into functions that operate on data passed to them or shared between them. Object-oriented code goes a step further by grouping data and behavior into units called objects. Within declarative programming, functional code builds programs out of expressions rather than state changes, whereas logic, query, and configuration languages describe the result without specifying how to compute it.

                      Programming Paradigms
          ┌─────────────────────┴──────────────────────┐
     imperative                                   declarative
     (say how)                                    (say what)
  ┌───────┴────────────┐                  ┌────────────┴───────────┐
procedural      object-oriented      functional      logic / query / config
Fortran (1957)  C++ (1985)           Lisp (1958)     regex (1968)
COBOL (1959)    Python (1991)        Haskell (1990)  SQL (1974)
C (1972)        Java (1995)          OCaml (1996)    HTML (1993)
Go (2009)       JavaScript (1995)    Clojure (2007)  Ansible (2012)
Rust (2010)     C# (2000)            Elixir (2012)   Terraform (2014)

Although Python is a multi-paradigm language, it’s primarily object-oriented. So, in this part we’ll cover the main ideas of object-oriented programming. We’ll show Python code, but the ideas aren’t Python specific. Our goal is to develop a SOLID understanding of OOP’s principles, so that we don’t repeat the same mistakes that have been made in the past.

Object-Oriented Programming

#

Objects are the fundamental unit of OOP. They’re instances of their blueprints, called classes. The state or data an object holds is stored in attributes, and the object’s behavior is defined by its methods.

Introductory OOP examples usually illustrate these ideas with small, self-contained classes.

The first example stores instance state and modifies it through a method:

class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

    def celebrate_birthday(self):
        self.age += 1

The next example places shared behavior in a base class while subclasses override the operation that varies:

class Shape:
    def __init__(self, name):
        self.name = name

    def area(self):
        pass

    def describe(self):
        return f"{self.name} has area {self.area()}"

class Circle(Shape):
    def __init__(self, radius):
        super().__init__("Circle")
        self.radius = radius

    def area(self):
        return 3.14 * self.radius ** 2

class Square(Shape):
    def __init__(self, side):
        super().__init__("Square")
        self.side = side

    def area(self):
        return self.side ** 2

$ python -i shapes.py
>>> Circle(10).describe()
'Circle has area 314.0'
>>> Square(10).describe()
'Square has area 100'

Here neither subclass defines the describe method. Both inherit it from the Shape base class, and the inherited method calls whichever area belongs to the instance it runs on.

Note that Shape is meant to be incomplete. Yet, Shape("mystery") constructs normally, and describe returns 'mystery has area None'. Programming languages often provide a mechanism for requiring the override. In Python it’s the abstract base class:

from abc import ABC, abstractmethod

class Animal(ABC):
    @abstractmethod
    def make_sound(self):
        raise NotImplementedError

class Cat(Animal):
    def make_sound(self):
        return "Meow"

class Dog(Animal):
    def make_sound(self):
        return "Woof"

class Jellyfish(Animal):
    # does not make a sound
    pass

$ python -i animals.py
>>> Cat().make_sound()
'Meow'
>>> Dog().make_sound()
'Woof'
>>> Animal()
Traceback (most recent call last):
  ...
TypeError: Can't instantiate abstract class Animal without an implementation for abstract method 'make_sound'
>>> Jellyfish()
Traceback (most recent call last):
  ...
TypeError: Can't instantiate abstract class Jellyfish without an implementation for abstract method 'make_sound'

The abstract base class itself can’t be instantiated, nor can a subclass that leaves make_sound unimplemented.

Another example is a class assembled from other objects, with a value computed from all of them:

class Item:
    def __init__(self, name, price):
        self.name = name
        self.price = price

class ShoppingCart:
    def __init__(self, items):
        self.items = items

    @property
    def total(self):
        return sum(item.price for item in self.items)

$ python -i cart.py
>>> cart = ShoppingCart([Item("keyboard", 80), Item("mouse", 25)])
>>> cart.total
105

These examples introduce the syntax of classes along with encapsulation, inheritance, composition, and polymorphism. Yet they don’t carry a concrete problem. In the age of AI, syntax by itself is the least important part of this material to focus on. So, in the rest of this part, we’ll cover the same ideas against problems that occur in production code.

Classes

#

Our running example is a rate limiter. Rate limiters are widely used in production systems such as APIs and alerting pipelines to control how frequently actions can occur. In alerting systems, for example, limiting notification frequency is important because recipients may mute channels that generate too many alerts.

A typical procedural implementation in Python keeps the state in a module-level deque and the logic in a function that reads and mutates it:

import collections
import time

sent_at = collections.deque()

def allow_send(max_per_hour):
    now = time.monotonic()
    while sent_at and now - sent_at[0] > 3600:
        sent_at.popleft()
    if len(sent_at) >= max_per_hour:
        return "blocked"
    sent_at.append(now)
    return "allowed"

$ python -i monitor.py
>>> allow_send(max_per_hour=2)
'allowed'
>>> allow_send(max_per_hour=2)
'allowed'
>>> allow_send(max_per_hour=2)
'blocked'

This works for one limit in a simple module, but when a program needs two or more rate limiters, procedural approaches with functions start to get out of hand. In Python, closures and generator coroutines can make it work in a relatively readable way, but the complexity increases nonetheless.

A class packages the state and its associated logic into a unit that can be instantiated multiple times.

import collections
import time

class RateLimiter:
    def __init__(self, max_per_hour):
        self.max_per_hour = max_per_hour
        self._events = collections.deque()

    def allow(self):
        now = time.monotonic()
        while self._events and now - self._events[0] > 3600:
            self._events.popleft()
        if len(self._events) >= self.max_per_hour:
            return "blocked"
        self._events.append(now)
        return "allowed"

Two things to note here. First, the __init__ method runs when an instance is created and sets up that instance’s attributes. The double underscore at the beginning and end of __init__ indicates it’s a special method in Python. These are called dunder or magic methods. We’ll devote part 2 of this chapter to these methods and their uses.

Second, self refers to the instance the method was called on, so self.max_per_hour and self._events belong to one limiter rather than to the class or the module. In other programming languages, this is often called this instead of self. Python uses self by convention rather than as a reserved keyword. So, we could have used any other name, including this, but it’s strongly recommended to stick with self for clarity and consistency.

Instantiating the class twice produces two independent limiters:

$ python -i rate_limiter.py
>>> pager = RateLimiter(max_per_hour=2)
>>> email = RateLimiter(max_per_hour=3)
>>> pager.allow()
'allowed'
>>> pager.allow()
'allowed'
>>> pager.allow()
'blocked'
>>> email.allow()
'allowed'

The pager limiter refuses its third send while the email limiter still accepts. Each instance carries its own max_per_hour and its own event history.

Attributes assigned in the class body behave differently from attributes assigned in __init__. They’re created once, when the class body executes, and every instance shares them. These are called class attributes.

For example, during a large incident, each channel can remain below its local limit while the combined volume still becomes excessive. A shared budget therefore needs state visible to every limiter instance. A class attribute can represent that shared state.

class RateLimiter:
    global_max_per_hour = 4
    # one deque for the whole class
    _global_events = collections.deque()

    def __init__(self, max_per_hour):
        self.max_per_hour = max_per_hour
        # one deque per instance
        self._events = collections.deque()

    @staticmethod
    def _expire(events, now):
        while events and now - events[0] > 3600:
            events.popleft()

    def allow(self):
        now = time.monotonic()
        self._expire(self._events, now)
        self._expire(RateLimiter._global_events, now)
        if len(self._events) >= self.max_per_hour:
            return "blocked by local limit"
        if len(RateLimiter._global_events) >= RateLimiter.global_max_per_hour:
            return "blocked by global limit"
        self._events.append(now)
        RateLimiter._global_events.append(now)
        return "allowed"

$ python -i rate_limiter.py
>>> pager = RateLimiter(max_per_hour=2)
>>> email = RateLimiter(max_per_hour=3)
>>> [pager.allow() for _ in range(3)]
['allowed', 'allowed', 'blocked by local limit']
>>> [email.allow() for _ in range(3)]
['allowed', 'allowed', 'blocked by global limit']

The blocked results now name the limit that applied. The third page is refused by the local limit of two. The two emails bring the shared total to four, so the third email is refused by the global limit while the email limiter has used only two of its own three.

Inside allow, the shared state is spelled with the class name, RateLimiter._global_events rather than self._global_events. Reading through self would find the same deque, because attribute lookup falls back from the instance to its class. The class name does two things: it marks each line that touches shared state, and it keeps an assignment from creating an instance attribute that shadows the class attribute.

For simplicity, let’s return "blocked" for both local and global limits. The rest of the example doesn’t need the distinction.

class RateLimiter:
    global_max_per_hour = 4
    # one deque for the whole class
    _global_events = collections.deque()

    def __init__(self, max_per_hour):
        self.max_per_hour = max_per_hour
        # one deque per instance
        self._events = collections.deque()

    @staticmethod
    def _expire(events, now):
        while events and now - events[0] > 3600:
            events.popleft()

    def allow(self):
        now = time.monotonic()
        self._expire(self._events, now)
        self._expire(RateLimiter._global_events, now)
        if len(self._events) >= self.max_per_hour:
            return "blocked"
        if len(RateLimiter._global_events) >= RateLimiter.global_max_per_hour:
            return "blocked"
        self._events.append(now)
        RateLimiter._global_events.append(now)
        return "allowed"

The @staticmethod decorator marks _expire as a method that receives neither self nor cls. So it has no implicit access to instance or class state, like a module-level function. Defining it as a static method namespaces the helper under RateLimiter.

We talked about dunder methods, but what about the single leading underscores in _expire and _events? Are these typos? They look a bit odd until you get used to them. The leading underscore in classes is another naming convention, which brings us to our next topic.

Encapsulation

#

Encapsulation is the practice of keeping an object’s internal state behind the methods that maintain its rules. One of the most important decisions you’ll make when designing your architecture is where to draw the encapsulation boundaries. These determine which parts of an object form its public interface and which remain implementation details.

In the rate limiter, _events is internal, since its contents are only meaningful in combination with max_per_hour and the expiry logic in allow. External code that appends to it or clears it breaks the rate guarantee without raising an error. So, a leading underscore tells the users of the class not to worry about how it works or how it’s implemented, and also not to mess with it.

Many object-oriented languages enforce this with access modifiers. C++, Java, and C# each provide public, protected, and private, and the compiler rejects access from outside the permitted scope.

Python doesn’t provide equivalent private or protected access restrictions. A single leading underscore marks an attribute as internal. A double leading underscore triggers name mangling: inside a class body, __events is rewritten to _RateLimiter__events. Since neither of these mechanisms restricts access, a motivated Python developer can always reach into the internals of a class. Encapsulation in Python is therefore a design decision rather than a language guarantee.

But why do we need both a single leading underscore and a double leading underscore? The answer is inheritance.

Inheritance

#

class RateLimiter:
    global_max_per_hour = 4
    # one deque for the whole class
    _global_events = collections.deque()

    def __init__(self, max_per_hour):
        self.max_per_hour = max_per_hour
        # one deque per instance
        self._events = collections.deque()

    @staticmethod
    def _expire(events, now):
        while events and now - events[0] > 3600:
            events.popleft()

    def allow(self):
        now = time.monotonic()
        self._expire(self._events, now)
        self._expire(RateLimiter._global_events, now)
        if len(self._events) >= self.max_per_hour:
            return "blocked"
        if len(RateLimiter._global_events) >= RateLimiter.global_max_per_hour:
            return "blocked"
        self._events.append(now)
        RateLimiter._global_events.append(now)
        return "allowed"

Our RateLimiter class currently does two jobs. This is also evident when we look at the allow method: it looks like it’s doing everything twice. Any program that needs only the local limit still carries the global limit, and the shared attributes exist whether or not anything uses them. Two independently varying concerns in one class is what the Single Responsibility Principle warns about, and a client depending on the half it doesn’t use is what the Interface Segregation Principle warns about.

Here we can separate the two with inheritance. The base class is a rate limiter without the global limit:

class RateLimiter:
    def __init__(self, max_per_hour):
        self.max_per_hour = max_per_hour
        self._events = collections.deque()

    @property
    def _remaining(self):
        return self.max_per_hour - len(self._events)

    def _expire(self, now):
        while self._events and now - self._events[0] > 3600:
            self._events.popleft()

    def _record(self, now):
        self._events.append(now)

    def allow(self):
        now = time.monotonic()
        self._expire(now)
        if self._remaining <= 0:
            return "blocked"
        self._record(now)
        return "allowed"

This is the same as the previous version without the global limit, except we did some refactoring. The _remaining property computes how many sends are left, _expire is back to being an instance method, and _record appends the current time to the instance’s deque. The “public” allow method reads more clearly as a result, because each step it takes is now named after the one thing it does. The refactoring buys readability and, as we’re about to see, somewhere for a subclass to intervene. Also, because these are meant to be internal methods, they’re prefixed with a single underscore as we saw before.

A property decorator exposes a computed value using attribute syntax. This allows us to have dynamic attributes. Here _remaining is recomputed on every access, but the caller reads it as a plain attribute. This is useful because an attribute can start as plain data and later become a property with computation or validation behind it, and existing reads keep working unchanged. Existing assignments keep working only if the property also defines a setter.

Next, we’ll create a GlobalRateLimiter subclass that inherits from the RateLimiter superclass and adds the shared budget on top of it:

class GlobalRateLimiter(RateLimiter):
    global_max_per_hour = 4
    _global_events = collections.deque()

    @classmethod
    def _global_remaining(cls):
        return cls.global_max_per_hour - len(cls._global_events)

    @property
    def _remaining(self):
        return min(super()._remaining, self._global_remaining())

    def _expire(self, now):
        super()._expire(now)
        while self._global_events and now - self._global_events[0] > 3600:
            self._global_events.popleft()

    def _record(self, now):
        super()._record(now)
        self._global_events.append(now)

Okay, so what exactly just happened here?

global_max_per_hour and _global_events are class attributes, exactly as they were before the split. They now live on GlobalRateLimiter rather than on RateLimiter, so a plain RateLimiter no longer carries them.

_global_remaining is a classmethod. The @classmethod decorator makes Python pass the class as the first argument instead of the instance, which is why the parameter is conventionally named cls rather than self. The shared budget belongs to the class, not to any one limiter.

_remaining overrides the base class property. super()._remaining evaluates the base class version, which knows only about this instance’s own events, and min combines it with what the shared budget allows. A subclass extends a property the same way it extends a method.

_expire and _record follow the same shape. Each calls super() to run the base class implementation on the instance’s own deque, then does the equivalent work on the shared deque. Without super(), each override would have to restate the local logic.

Note that GlobalRateLimiter never defines allow. The inherited allow runs unchanged. Every step it takes is a call on self, and self is a GlobalRateLimiter, so each of those calls finds the subclass version:

    def allow(self):
        now = time.monotonic()
        self._expire(now)  # subclass: expires both deques
        if self._remaining <= 0:  # subclass: min(local, global)
            return "blocked"
        self._record(now)  # subclass: records in both deques
        return "allowed"

Our new inheritance-based approach works as expected. The pager is refused by its own limit of two. The emails bring the shared total to four, so the third is refused even though that limiter has used only two of its own three. A plain RateLimiter is unaffected by any of it, because the shared budget now belongs to the subclass.

$ python -i rate_limiter.py
>>> pager = GlobalRateLimiter(max_per_hour=2)
>>> email = GlobalRateLimiter(max_per_hour=3)
>>> [pager.allow() for _ in range(3)]
['allowed', 'allowed', 'blocked']
>>> [email.allow() for _ in range(3)]
['allowed', 'allowed', 'blocked']
>>> GlobalRateLimiter._global_remaining()
0
>>> RateLimiter(max_per_hour=3).allow()
'allowed'

At the end of the encapsulation section, we said the reason for having both underscores is inheritance. A subclass stores its attributes in the same instance as its base class, so if both used the name _events, the subclass’s assignment would replace the base class’s deque. Name mangling prefixes the class name, so each class gets its own attribute.

Going back to our inheritance approach, we saw that inheritance works. Yet, it introduces several costs.

First, every method in GlobalRateLimiter has to call super(). Forgetting one of those calls will eventually happen. This imposes a mental cost on everyone who extends the class.

Second, inheritance required changes to the base class. The three hooks didn’t exist until the subclass needed them, so extension first required modification. This illustrates a practical difficulty with the Open/Closed Principle. Useful extension points are often known only after new requirements appear.

Third, a subclass should be substitutable for its base class, which is known as the Liskov Substitution Principle. This doesn’t require identical behavior, since that would rule out overriding. It requires the subclass to keep the guarantees the base class made. A fresh RateLimiter(max_per_hour=5) permits five immediate sends before its local capacity is exhausted. GlobalRateLimiter doesn’t preserve that behavior.

$ python -i rate_limiter.py
>>> def drain(limiter):
...     count = 0
...     while limiter.allow() == "allowed":
...         count += 1
...     return count
... 
>>> drain(RateLimiter(max_per_hour=5))
5
>>> hog = GlobalRateLimiter(max_per_hour=10)
>>> [hog.allow() for _ in range(4)]
['allowed', 'allowed', 'allowed', 'allowed']
>>> drain(GlobalRateLimiter(max_per_hour=5))
0

The second limiter was constructed the same way and permits nothing. Its capacity was spent by hog, which is an object the caller didn’t see.

The reason all of this is awkward is that the code says a global rate limiter is a rate limiter. But it’s better to model this differently. A rate limiter has limits, and the number of them is a property of how it was configured. That’s composition, the underutilized counterpart of inheritance.

Composition

#

Composition provides a more flexible approach. The limit mechanics move into an object of their own:

class Limit:
    def __init__(self, max_per_hour):
        self.max_per_hour = max_per_hour
        self._events = collections.deque()

    @property
    def remaining(self):
        return self.max_per_hour - len(self._events)

    def expire(self, now):
        while self._events and now - self._events[0] > 3600:
            self._events.popleft()

    def record(self, now):
        self._events.append(now)

Note that remaining, expire, and record are now public, because RateLimiter has a Limit rather than being one. A RateLimiter always has a local limit of its own, and it takes any number of further limits that were created elsewhere:

class RateLimiter:
    def __init__(self, max_per_hour, *shared_limits):
        self.local_limit = Limit(max_per_hour)
        self.limits = (self.local_limit, *shared_limits)

    @property
    def remaining(self):
        return min(limit.remaining for limit in self.limits)

    def allow(self):
        now = time.monotonic()
        for limit in self.limits:
            limit.expire(now)
        if self.remaining <= 0:
            return "blocked"
        for limit in self.limits:
            limit.record(now)
        return "allowed"

The shared budget is a Limit like any other, and two limiters share it when they’re given the same one:

$ python -i rate_limiter.py
>>> global_limit = Limit(max_per_hour=4)
>>> pager = RateLimiter(2, global_limit)
>>> email = RateLimiter(3, global_limit)
>>> [pager.allow() for _ in range(3)]
['allowed', 'allowed', 'blocked']
>>> [email.allow() for _ in range(3)]
['allowed', 'allowed', 'blocked']
>>> email.local_limit.remaining
1

GlobalRateLimiter is gone because the shared limit no longer needs a class. Under inheritance it was a class attribute, so every instance got it whether it used it or not, and one process could hold only one budget. Under composition a budget is a Limit object, and two rate limiters can share it. So, a third limit is now an argument rather than a class.

$ python -i rate_limiter.py
>>> first_limit = Limit(max_per_hour=4)
>>> second_limit = Limit(max_per_hour=3)
>>> pager = RateLimiter(2, first_limit, second_limit)
>>> email = RateLimiter(3, first_limit, second_limit)
>>> [pager.allow() for _ in range(2)]
['allowed', 'allowed']
>>> [email.allow() for _ in range(2)]
['allowed', 'blocked']
>>> first_limit.remaining, second_limit.remaining, email.local_limit.remaining
(1, 0, 2)

Polymorphism

#

Our last concept is polymorphism, which is the property that one call site produces different behavior depending on the object it receives.

We’ve seen this with both inheritance and composition. The inherited allow called self._expire and reached the subclass implementation. The composed allow calls limit.expire on whatever collaborators it was given.

Recap

#

Object-oriented programming is one paradigm among several.

One of its central uses is keeping state together with the behavior that manages it, in units that can be instantiated independently. Classes define those units.

Encapsulation separates the public interface from implementation details. We saw that Python relies on conventions rather than enforced access restrictions.

Inheritance shares implementation and establishes a subtype relationship, but it can reduce locality of reasoning. Understanding a subclass may require reading both the base implementation and the subclass’s overrides. Overrides must cooperate with the base implementation, the base class must expose appropriate extension points, and independently varying combinations tend to require additional types.

Composition assembles behavior from components selected when the object is constructed rather than when the class is defined. New combinations can therefore be expressed through different component configurations.

Both designs can use polymorphism, allowing a call site to operate through an interface while the concrete object determines the behavior.

In part 2, we’ll understand Python’s OOP philosophy and why dunder or magic methods are central to making objects work naturally with the language. We’ll also trace method lookup through the descriptor protocol, which explains how self is bound automatically and underlies properties, class methods, static methods, and super(). See you there!

Portraits from Wikimedia Commons, cropped and desaturated: Robert C. Martin by Angelacleancoder, CC BY-SA 4.0; Bertrand Meyer by David Monniaux, CC BY-SA 3.0; Barbara Liskov by Kenneth C. Zirkel, CC BY-SA 3.0.