Threads and concurrency

A web server handles hundreds of requests at once. A desktop app keeps its interface responsive while saving a file. A data job uses all 16 cores of a machine. All of these rely on concurrency — several things making progress at the same time — and most of them use threads.

Threads versus processes

A process can contain several threads: independent sequences of execution that the scheduler can run on different cores at the same time. What they share, and what they don’t, is the key to everything that follows:

Shared by all threads

The address space — heap, global variables and code — plus open files and network sockets.

Private to each thread

Its stack (local variables and call frames), its registers and program counter, and its scheduling state.

Threads versus processes

Threads are cheaper to create and communicate by simply reading and writing shared memory. Processes are isolated and must communicate through the kernel — safer, but heavier.

Sharing memory makes threads fast to coordinate. It is also what makes them dangerous.

An increment is not one step

Consider the most innocent line of code: counter = counter + 1. To the CPU it is three separate steps:

1. load the value of counter from memory into a register;
2. add 1 to the register;
3. store the register back to memory.

With two threads running this at the same time, the scheduler — or two cores running truly in parallel — can interleave those steps in any order. Step through one unlucky interleaving, then switch on the mutex and run it again.

Race conditions and critical sections

A race condition is a bug whose outcome depends on the timing of threads. It may appear once in a million runs — only under load, only on one customer’s machine, never under the debugger. That makes races among the hardest bugs in software.

The code that touches shared data is a critical section. The fix is mutual exclusion: guaranteeing that only one thread is inside a critical section at a time. The basic tool is a mutex (lock):

import threading

counter = 0
lock = threading.Lock()

def record_visit():
    global counter
    with lock:            # only one thread at a time past this line
        counter += 1      # read, add, write — now safe

The with statement releases the lock automatically, even if the code inside raises an exception — forgetting to unlock is one of the classic concurrency bugs.

The price of locking

Locks solve races but introduce new problems:

• Contention. If many threads need the same lock, they queue up and the program becomes effectively single-threaded at that point.
• Deadlock. Thread A holds lock X and waits for Y; thread B holds Y and waits for X. Both wait forever. The standard defence is to always acquire multiple locks in the same order.
• Complexity. Every shared variable needs a clear rule about which lock protects it.

Step through a deadlock, then fix it with a lock order:

Contention is subtler: nothing hangs, the program just stops getting faster. How much can more threads help when part of the work must happen under one lock?

The best strategy is to share less. Give each thread its own data and combine results at the end, pass messages through thread-safe queues instead of sharing objects, and prefer immutable data, which can be shared freely because nobody can change it.

In the final lecture of this module, we turn to memory: why some reads take a nanosecond and others take a million times longer — and how every process gets its own private view of RAM.