Thursday, November 5, 2026

Week 1: Python Data Model and Internals

Overview

By the end of this tutorial you will be able to predict what Python does with memory and references whenever you assign, copy, mutate, or pass an object.

This is the first topic of Week 1 (Python Data Model and Internals). Everything here is runnable on Python 3.11 or newer; the outputs shown were produced on Python 3.13. Type the examples into a REPL instead of only reading them.

You will learn to:

  • Describe every Python value by its identity, type, and value
  • Explain why a variable is a name bound to an object, not a box holding data
  • Tell mutable and immutable types apart, and predict when += mutates or rebinds
  • Choose between aliasing, shallow copies, and deep copies
  • Explain how arguments are passed to functions, and avoid the mutable default argument trap
  • Explain how CPython frees memory with reference counting, and measure object sizes correctly

Prerequisites: basic Python (variables, lists, dicts, functions, classes).

Time: about 90 minutes for reading and experiments, plus 30 minutes for the mini project and exercises.

1. Everything is an object

Every value in Python, from 42 to a function to a class, is an object with exactly three properties: identity, type, and value.

Property

Meaning

Changes?

How to inspect

Identity

Which object it is; in CPython, its memory address

Never, for the object's lifetime

id(obj)

Type

What kind of object it is; decides which operations work

Never

type(obj)

Value

The data it holds

Only if the type is mutable

print(obj), repr(obj)

x = [1, 2, 3]                                                       
y = x                                                               
                                                                    
print(y is x)           # True:  same identity                      
print(y == [1, 2, 3])   # True:  equal value                        
print(y is [1, 2, 3])   # False: equal value, but a different object

is versus ==

  • == asks "do these two objects have equal values?" It calls __eq__, which a class can customize.
  • is asks "are these the very same object?" It compares identities and cannot be overridden.

Use == for values. Use is only for None and for sentinel objects you created yourself, as in if result is None:.

Functions, classes, and types are objects too

def greet(): ...                                    
                                                    
print(type(greet))              # <class 'function'>
print(type(int))                # <class 'type'>    
print(type(type))               # <class 'type'>    
print(isinstance(greet, object))   # True           
print(type(None))               # <class 'NoneType'>

Because functions are objects, you can store them in lists, pass them as arguments, and give them attributes. Because classes are objects (instances of type), you can create them at runtime. Later weeks build on both facts.

Identity in CPython

In CPython, id(obj) is the object's address in memory, so hex(id(obj)) looks like 0x7f3c.... That is an implementation detail: other Python implementations number objects differently, and an id may be reused after an object is freed. Treat id() as an opaque label that is unique among live objects.

Try it: create two equal lists, then print id() for each. Then bind a third name to the first list and print its id again. Which ids match, and why?

2. Names and references

In Python a variable is a name bound to an object, so assignment attaches a label to an existing object instead of copying data into a box.

Three rules explain nearly all reference behaviour:

  1. name = expression evaluates the expression to an object, then binds the name to it
  2. Binding never copies the object, so two names bound to one object are aliases
  3. Mutating an object (calling append) and rebinding a name (b = [9]) are different operations
a = [1, 2, 3]                                           
b = a                # a second name for the SAME list  
b.append(4)          # mutates the one list             
print(a, a is b)     # [1, 2, 3, 4] True                
                                                        
b = [9]              # rebinding: b now names a NEW list
print(a, b, a is b)  # [1, 2, 3, 4] [9] False           

Assignment binds names to objects; it never copies them.





Left: both names point at one list, so append is visible through either name. Right: b = [9] moves only the name b to a new object; a and the first list are unaffected.

Binding happens in more places than =

The same "attach a name to an object" step runs for loops, imports, function and class definitions, and function parameters.

for i in range(3): ...    # binds i on each pass                  
import math               # binds math to the module object       
def f(): ...              # binds f to a function object          
class C: ...              # binds C to a class object             
with open("x.txt", "w") as fh: ...   # binds fh to the file object

Why swapping needs no temporary

Python evaluates the whole right-hand side first, then binds the names, so labels can trade objects directly.

p, q = 1, 2         
p, q = q, p         
print(p, q)    # 2 1

Nothing moved in memory; the names p and q simply swapped which object each one points to.

Names live in namespaces

A namespace is a mapping from names to objects, which you can inspect: globals() returns the module's names as a dict, and locals() returns a function's. del name removes the entry from the mapping and leaves the object alone, which section 5 returns to.

Try it: run print(globals()) in a fresh REPL, then create a variable and print again to see the new entry appear.

3. Mutability

An object is mutable if its value can change in place without changing its identity, and immutable if it cannot.

Immutable

Mutable

int, float, complex, bool

    list

str, bytes

    dict, set

tuple, frozenset

    bytearray, collections.deque

None, range

    most instances of classes you write

An immutable object never changes; operations on it return new objects.

s = "abc"                                                      
s2 = s.upper()                                                 
print(s, s2)      # abc ABC: s is untouched, s2 is a new string

Immutability is shallow

A tuple fixes which objects it contains, but it cannot stop a mutable object inside it from changing.

t = ([1, 2], "fixed")                                                         
t[0].append(3)                                                                
print(t)          # ([1, 2, 3], 'fixed')                                      
                                                                              
t[0] = []         # TypeError: 'tuple' object does not support item assignment

The tuple still holds the same list object, so its identities did not change; only the list's value did.

+= mutates or rebinds, depending on the type

The augmented assignment a += b first tries a.__iadd__(b), which mutable types implement as an in-place update. If the type has no __iadd__, Python falls back to a = a + b, which creates a new object and rebinds the name.

l = [1, 2]; m = l
l += [3]              # list.__iadd__ mutates in place
print(m)              # [1, 2, 3]: m sees the change

s = (1, 2); u = s
s += (3,)             # tuples have no __iadd__: s is rebound to a new tuple
print(u, s)           # (1, 2) (1, 2, 3)

l = [1, 2]; m = l
l = l + [3]           # list + list builds a NEW list
print(m, l)           # [1, 2] [1, 2, 3]

So l += x and l = l + x look alike but differ for lists when another name points to the same list.

Hashability

Dictionary keys and set members must be hashable: they need a hash that never changes while they are in the container. Immutable built-ins qualify; mutable ones do not.

print(hash((1, 2)) == hash((1, 2)))   # True

{[1]: 0}              # TypeError: unhashable type: 'list'
hash(([1], 2))        # TypeError: unhashable type: 'list'

A tuple is hashable only if everything inside it is hashable, which is shallow immutability showing up again.

Why immutability exists

  • Safety: an immutable object can be shared freely between names, functions, and threads without anyone changing it behind your back
  • Hashing: only objects with a stable value can be dict keys or set members
  • Optimization: CPython can reuse and cache immutable objects, as section 5 shows

The cost is that building a large string with repeated s += piece can copy the string each time, though CPython sometimes optimizes it; collect pieces in a list and call "".join(pieces) once instead.

4. Copying and passing objects

Assignment never copies, so any time you need an independent object you must copy explicitly, and you must choose how deep the copy goes.

Shallow copy versus deep copy

  • A shallow copy makes a new outer container but reuses the same inner objects
  • A deep copy recursively copies everything, so nothing is shared
import copy

matrix = [[1, 2], [3, 4]]
shallow = copy.copy(matrix)
deep = copy.deepcopy(matrix)

matrix[0].append(99)
print(shallow[0])                 # [1, 2, 99]: inner list is shared
print(deep[0])                    # [1, 2]:     fully independent
print(shallow[0] is matrix[0])    # True
print(deep[0] is matrix[0])       # False

These are all shallow copies of a list: lst[:], lst.copy(), list(lst), and copy.copy(lst). For dicts the equivalents are d.copy(), dict(d), and copy.copy(d).

d = {"k": [1]}
e = dict(d)
e["k"].append(2)      # mutates the shared inner list
e["z"] = 0            # changes only e
print(d)              # {'k': [1, 2]}

copy.deepcopy also handles cycles: it remembers what it has already copied, so a list that contains itself copies without looping forever.

c = [1]; c.append(c)
d = copy.deepcopy(c)
print(d[1] is d, d is c)   # True False

Deep copies cost time and memory proportional to the whole object graph, so use them deliberately. For your own classes you can customize copying with __copy__ and __deepcopy__.

The nested list trap

Repeating a list with * repeats references to one inner object, not copies of it.

grid = [[0] * 3] * 3
grid[0][0] = 1
print(grid)    # [[1, 0, 0], [1, 0, 0], [1, 0, 0]]: one row, three names

grid = [[0] * 3 for _ in range(3)]    # a comprehension builds three separate rows
grid[0][0] = 1
print(grid)    # [[1, 0, 0], [0, 0, 0], [0, 0, 0]]

[0] * 3 is safe because integers are immutable; the trap only appears when the repeated element is mutable.

How arguments are passed

Python passes arguments by object reference, sometimes called "call by sharing": the function's parameter becomes a new name bound to the same object the caller passed. This follows directly from section 2.

  • Mutating the object inside the function is visible to the caller, because it is one object
  • Rebinding the parameter inside the function is invisible to the caller, because it only moves a local name
def mutate(items):
    items.append("x")     # changes the shared list

def rebind(items):
    items = ["new"]       # moves the local name only

data = ["a"]
mutate(data); print(data)    # ['a', 'x']
rebind(data); print(data)    # ['a', 'x']: unchanged

It is neither "pass by value" nor "pass by reference" in the C++ sense. If a function should not alter its input, copy it inside the function or document that it mutates.

The mutable default argument trap

A default value is evaluated once, when the def statement runs, and the resulting object is stored on the function. A mutable default is therefore shared by every call that omits the argument.

def add_item(item, bucket=[]):    # bug: one list for all calls
    bucket.append(item)
    return bucket

print(add_item(1))           # [1]
print(add_item(2))           # [1, 2]: surprise
print(add_item.__defaults__) # ([1, 2],)

def add_item_fixed(item, bucket=None):
    if bucket is None:
        bucket = []          # a fresh list on every call
    bucket.append(item)
    return bucket

print(add_item_fixed(1), add_item_fixed(2))   # [1] [2]

The None sentinel pattern is the standard fix; linters such as ruff flag the buggy form.

5. Memory management

CPython frees most objects the instant nothing refers to them, by keeping a reference count on every object.

Reference counting

Each object stores how many references point to it: names, list slots, dict entries, and function arguments all count. When the count drops to zero, CPython frees the object immediately. sys.getrefcount(obj) reports the count, and it is always one higher than you expect because passing obj as the argument creates a temporary reference.

import sys

obj = object()
print(sys.getrefcount(obj))   # 2: the name obj + the call's temporary reference
alias = obj
print(sys.getrefcount(obj))   # 3
holder = [obj]
print(sys.getrefcount(obj))   # 4: a list slot is a reference too
del alias
print(sys.getrefcount(obj))   # 3
holder.clear()
print(sys.getrefcount(obj))   # 2

del name removes a name, not an object. The object is freed only when its last reference goes away.

You can watch an object die with a weak reference, which does not add to the count:

import weakref

class Thing: pass

th = Thing()
w = weakref.ref(th)
print(w() is th)      # True: still alive
del th                # last strong reference gone
print(w() is None)    # True: freed immediately

Three things raise or lower a count: binding a name, storing the object in a container, and passing it into a call; unbinding, removing from a container, and returning from a call reverse them.

Reference cycles. Two objects that refer to each other keep each other's count above zero even when nothing else can reach them. CPython runs a separate cyclic garbage collector (the gc module) to find and free such groups. Week 1's third topic covers it in depth.

Small-integer caching and string interning

CPython reuses some immutable objects instead of creating new ones. These are optimizations, not language guarantees, so never use is to compare numbers or strings.

a = 256; b = 256
print(a is b)                  # True: small ints (-5 to 256) are cached

x = int("1000"); y = int("1000")
print(x is y, x == y)          # False True: two separate objects, equal values

s1 = "hello"; s2 = "hel" + "lo"
print(s1 is s2)                # True: the compiler folded the constant and shared it

s3 = "".join(["hel", "lo"])
print(s1 is s3, s1 == s3)      # False True: built at runtime, a new object
print(sys.intern(s3) is sys.intern("hello"))   # True: intern() returns one shared copy

sys.intern is useful when you hold millions of repeated strings, such as column names or categories, because it stores each distinct string once and lets you compare by identity.

From Python 3.12, objects such as None, True, and small integers are immortal: their reference counts are fixed at a huge value and never change, so sys.getrefcount(None) returns a very large number rather than a small one.

How big is an object?

sys.getsizeof(obj) returns the size of the object itself in bytes, not the size of the objects it refers to. A container holds references, so its own size depends only on how many slots it has. These figures are from 64-bit CPython 3.13 and vary by version and platform:

import sys

print(sys.getsizeof([]), sys.getsizeof([1, 2, 3]), sys.getsizeof(list(range(1000))))
# 56 88 8056
print(sys.getsizeof(1), sys.getsizeof("a"), sys.getsizeof((1, 2, 3)), sys.getsizeof({}))
# 28 42 64 64

big = "x" * 1_000_000
items = [big] * 10
print(sys.getsizeof(items), sys.getsizeof(big))
# 136 1000041: ten references to one 1 MB string cost 136 bytes, not 10 MB

The last line shows two ideas at once: a list stores pointers, and aliasing saves memory. A list of ten names for one big object costs almost nothing extra, but a list of ten copies would cost ten times as much.

Lists also over-allocate so that append is fast on average. Watching the size jump shows it:

l = []; last = sys.getsizeof(l); jumps = []
for i in range(20):
    l.append(i)
    size = sys.getsizeof(l)
    if size != last:
        jumps.append((len(l), size)); last = size
print(jumps)   # [(1, 88), (5, 120), (9, 184), (17, 248)]

The size grows in steps, not on every append, so the list has spare capacity most of the time.

To measure an object together with everything it references, walk the graph and count each object once:

def deep_size(obj, seen=None):
    seen = set() if seen is None else seen
    if id(obj) in seen:               # already counted: shared object
        return 0
    seen.add(id(obj))
    size = sys.getsizeof(obj)
    if isinstance(obj, dict):
        size += sum(deep_size(k, seen) + deep_size(v, seen) for k, v in obj.items())
    elif isinstance(obj, (list, tuple, set, frozenset)):
        size += sum(deep_size(i, seen) for i in obj)
    return size

print(deep_size([big] * 10))             # 1000177: the shared string is counted once
print(deep_size([big, "x" * 1_000_000])) # 2000154: two different strings

This helper handles only the common built-in containers; it ignores instance attributes. For real memory investigations use tracemalloc, which tracks allocations by source line:

import tracemalloc

tracemalloc.start()
data = [str(i) * 10 for i in range(100_000)]
current, peak = tracemalloc.get_traced_memory()
print(f"{current / 1e6:.1f} MB in use, peak {peak / 1e6:.1f} MB")

for stat in tracemalloc.take_snapshot().statistics("lineno")[:3]:
    print(stat)    # the three lines that allocated the most memory
tracemalloc.stop()

Freed memory is not always returned to the operating system right away, because CPython's allocator keeps free blocks for reuse, so a process's size in a task manager can stay high after objects die.

6. Mini project, exercises, and checklist

The mini project turns the ideas above into two small debugging tools, and the exercises check that your mental model predicts real output.

Mini project: an alias detector

Save this as aliases.py. It uses id() to find names, or list positions, that share one object.

from collections import defaultdict


def find_aliases(namespace):
    """Group the names in `namespace` that refer to the same object."""
    groups = defaultdict(list)
    for name, value in namespace.items():
        groups[id(value)].append(name)
    return [sorted(names) for names in groups.values() if len(names) > 1]


def shared_items(container):
    """Return the positions in `container` that hold the same object."""
    seen = defaultdict(list)
    for index, item in enumerate(container):
        seen[id(item)].append(index)
    return [positions for positions in seen.values() if len(positions) > 1]

Save this as test_aliases.py, then run pip install pytest and pytest -q. The four tests passed on Python 3.13.

from aliases import find_aliases, shared_items


def test_find_aliases():
    a = [1, 2]
    b = a
    c = [1, 2]
    assert find_aliases({"a": a, "b": b, "c": c}) == [["a", "b"]]


def test_equal_but_separate_objects_are_not_aliases():
    assert find_aliases({"a": [1], "b": [1]}) == []


def test_small_ints_are_aliases():
    assert find_aliases({"x": 1, "y": 1}) == [["x", "y"]]


def test_shared_items_detects_the_grid_bug():
    bad = [[0] * 3] * 3
    good = [[0] * 3 for _ in range(3)]
    assert shared_items(bad) == [[0, 1, 2]]
    assert shared_items(good) == []

Three lessons are packed into these tests:

  • Equal values are not aliases; only the same object is (test 2)
  • Cached small integers are aliases, so a naive detector reports false alarms for immutable values (test 3)
  • shared_items catches the [[0] * 3] * 3 bug, which is the most common aliasing mistake in practice (test 4)

Extend it: make shared_items ignore immutable items so cached integers do not show up, then make it recurse into nested lists and report the path to each shared object, such as grid[0] is grid[1].

Predict the output

Write down your answer for each snippet before running it.

# 1
a = [1, 2, 3]
b = a
b += [4]
print(a)

# 2
x = (1, 2)
y = x
x += (3,)
print(y)

# 3
rows = [[0] * 2] * 2
rows[0][0] = 1
print(rows)

# 4
def f(n, acc=[]):
    acc.append(n)
    return acc
f(1)
print(f(2))

# 5
t = ([1], [2])
t[0].append(9)
print(t)

# 6
s = "py"
u = s
s += "thon"
print(u)

# 7
d = {"k": [1]}
e = dict(d)
e["k"].append(2)
e["z"] = 0
print(d)

#

Output

Why

    1

[1, 2, 3, 4]

+= calls list.__iadd__, which mutates the one shared list

    2

(1, 2)

Tuples have no __iadd__, so x is rebound and y keeps the old tuple

    3

[[1, 0], [1, 0]]

* repeats a reference, so both rows are one list

    4

[1, 2]

The default list is created once and shared across calls

    5

([1, 9], [2])

Tuple immutability is shallow; the inner list can still change

    6

py

Strings are immutable, so += rebinds s and u keeps the old string

    7

{'k': [1, 2]}

dict(d) is a shallow copy: the inner list is shared, but the new key lands only in e

Hands-on exercises

  1. Refcount tracker: write a script that prints sys.getrefcount for one object after each of these steps: create it, bind a second name, add it to a list, add it to a dict, then undo each step. Explain every number.
  2. Copy depth: build a dict that contains a list of dicts. Make a shallow and a deep copy, mutate the original at three different depths, and record which copy changes each time.
  3. Spot the sharing: use id() to show that lst[:], list(lst), and lst.copy() all produce a new outer list whose items are the same objects as the original's.
  4. Fix the bug: this function has a hidden aliasing bug; find it and fix it.
    def make_teams(names, size=2, teams=[]):
        for i in range(0, len(names), size):
            teams.append(names[i:i + size])
        return teams
  5. Memory measurement: use deep_size from section 5 to compare 100,000 copies of one string held in a list against 100,000 distinct strings, then explain the difference.
  6. Intern it: generate 1,000,000 short strings from a small set of 10 values using "".join, measure memory with tracemalloc, then repeat with sys.intern. Report the saving.

Common pitfalls

Pitfall

Fix

Comparing numbers or strings with is

Use ==; reserve is for None and sentinels

Mutable default arguments

Default to None and create the object inside the function

[[0] * n] * m to make a grid

Use [[0] * n for _ in range(m)]

Assuming b = a copies

It binds a second name; call copy or deepcopy

Using copy.copy on nested data and expecting independence

Use copy.deepcopy, or copy each level you change

Treating l += x and l = l + x as identical

The first mutates in place; the second builds a new list

Reading sys.getsizeof as total memory

It excludes referenced objects; walk the graph or use tracemalloc

Relying on id() values or small-int caching

They are CPython implementation details

Checklist

  • I can state the identity, type, and value of any object and show them in code
  • I can explain why b = a never copies data
  • I can say, for any built-in type, whether it is mutable and whether it is hashable
  • I can predict whether += mutates or rebinds
  • I can choose between aliasing, a shallow copy, and a deep copy, and justify the choice
  • I can explain what happens to the caller's object when a function mutates or rebinds a parameter
  • I can explain how reference counting frees an object, and why getrefcount reads high by one
  • I can measure an object's real memory use, and know why getsizeof is only a start
  • My alias detector passes all four tests

Next topic: dunder methods, operator overloading, and custom containers, where you will define how your own objects behave under ==, +, len(), and iteration.


Advanced Python for Professionals: Syllabus

No comments:

Post a Comment

Advanced Functions and Object-Oriented Python

Level: Intermediate to advanced Suggested time: 5–7 hours Python: 3.10+ (examples use standard-library features) This week moves beyond w...