Matrices as transformations
A matrix is a machine that transforms vectors — stretching, rotating, shearing or flattening space. See matrices move grids in 2D and cubes in 3D, read the determinant as a change in area and volume, and understand matrix multiplication as a neural-network layer.
In this lecture you will
- Read the columns of a matrix as the new positions of the basis vectors
- Recognize stretches, rotations, shears, reflections and projections
- Interpret the determinant as the change in area or volume
- Multiply matrices and vectors, and explain why order matters
If vectors are the nouns of machine learning, matrices are the verbs. A matrix is a grid of numbers, but the useful way to think of one is as a transformation: feed it a vector and it returns a new vector. Every layer of a neural network, every rotation in computer graphics and every step of PCA is a matrix doing this.
Columns tell you everything
In two dimensions, every vector is a combination of the two basis vectors î = (1, 0) and ĵ = (0, 1). A matrix only needs to say where those two land — those are its columns — and every other vector follows along. The grid of the plane stays evenly spaced and parallel: the transformation is linear.
The teal arrow is the first column (where î lands), the amber arrow the second (where ĵ lands). Watch the letter F: a reflection mirrors it, and the ‘flatten’ preset squashes everything onto a line.
To transform any vector, multiply: take x copies of the first column and y copies of the second.
The determinant
The determinant measures how much a transformation scales area (in 2D) or volume (in 3D). For a 2 × 2 matrix it is ad − bc. A determinant of 2 doubles areas; a negative one also flips space over, like a mirror; and a determinant of zero means space has been flattened into a lower dimension. Information is lost — many inputs land on the same output — so the transformation cannot be undone.
- The identityA unit cube with a grid of points inside it. The three arrows are the basis vectors — the columns of the identity matrix.
- StretchA diagonal matrix stretches x by 1.8 and squashes z to 0.55. The volume changes by 1.8 × 1 × 0.55 = 0.99 — almost unchanged.
- ShearA shear slides layers of space sideways, like a deck of cards. The cube leans over, but its volume stays exactly the same: the determinant is 1.
- RotationA rotation turns space without stretching it. Lengths, angles and volume are all preserved.
- ProjectionSetting the middle row to zero sends every height to 0 and flattens the cube onto the floor. The determinant is 0: the heights are lost for good.
- Matrix
- A rectangular grid of numbers; as a transformation, its columns are the images of the basis vectors.
- Linear transformation
- A transformation that keeps grid lines straight, parallel and evenly spaced, with the origin fixed.
- Determinant
- The factor by which a square matrix scales areas or volumes; zero means the matrix cannot be inverted.
- Inverse
- The matrix that undoes a transformation. It exists only when the determinant is not zero.
- Identity
- The matrix that leaves every vector unchanged, with ones on the diagonal.
Matrix multiplication is a neural-network layer
Multiplying a matrix by a vector computes one dot product per row. That is precisely a dense neural-network layer: each row of the weight matrix holds one neuron’s weights, and each output is that neuron’s weighted sum, before the activation function.
- Shapes firstA weight matrix with 3 rows and 4 columns meets an input with 4 numbers. The inner sizes match, so the output has 3 numbers — one per row.
- Row times columnThe first output is the dot product of the first row with the input: multiply pairs and add.
- Every row, same inputThe other rows do the same with their own weights. Each row picks out a different pattern in the input.
- A whole batchStack several inputs as columns and the same rule produces all the outputs at once. GPUs are built to do exactly this, very fast.
- Order mattersApplying A and then B is the product BA. Rotating then stretching gives a different shape from stretching then rotating: in general AB ≠ BA.
import numpy as np
W = np.array([[2, 0, -1, 1],
[1, 3, 0, -2],
[0, 1, 2, 1]])
x = np.array([1, 3, 2, -1])
print(W @ x) # [-1 12 6] — one dot product per row
X = np.array([[1, 0, 2], [3, 1, 0], [2, -1, 1], [-1, 2, 1]])
print(W @ X) # a batch of three inputs at onceSome vectors are special: a matrix only stretches them, without turning them. Those eigenvectors reveal the main directions of a transformation — and, applied to data, the main directions of variation. That is the next lecture.
Key takeaways
- A matrix transforms space; its columns show where the basis vectors land, and every other vector follows.
- The determinant is the factor by which areas or volumes change — negative when space is flipped, zero when it is flattened and information is lost.
- Matrix × vector is a set of dot products, one per row — exactly what a neural-network layer computes before its activation.
- Applying transformations one after another multiplies their matrices, and the order matters.
Check your understanding
4 questions · answers explainedQ1A 2 × 2 matrix has columns (2, 0) and (0, 3). What does it do?
Q2A matrix has determinant 0. What follows?
Q3A layer multiplies a 128 × 512 weight matrix by an input vector. How long must the input be, and how long is the output?
Q4Why does the order of matrix multiplication matter?