Vectors and similarity
Machine learning turns everything — customers, images, sentences — into vectors. Learn what a vector is, how to add, scale and measure one, and how the dot product, cosine similarity and distance tell you how alike two things are.
In this lecture you will
- Describe a vector as an arrow and as a list of numbers
- Add vectors, multiply them by numbers and compute their length
- Compute a dot product two ways and interpret its sign
- Measure similarity with cosine similarity and Euclidean distance
Every machine-learning model starts by turning the world into numbers. A customer becomes a list of numbers, so does an image, so does a sentence. Such a list is a vector, and almost everything in machine learning — models, embeddings, gradients — is built from operations on vectors. This lecture covers the handful of operations you will use constantly.
What is a vector?
A vector is an ordered list of numbers, such as v = (2, 1, 2). Geometrically it is an arrow from the origin: go 2 along x, 1 along y and 2 along z. The same list can also be read as a point — the tip of the arrow. Rotate the scene and step through the basic operations.
- An arrow and a listThe vector v = (2, 1, 2) is an arrow from the origin. The dashed lines show its three components along x, y and z.
- AdditionTo add w to v, add the components: (2, 1, 2) + (1, 1.5, −1) = (3, 2.5, 1). Geometrically, place w at the tip of v; the sum goes from the origin to the new tip.
- ScalingMultiplying by a number scales every component. 1.5v is longer, −0.5v is shorter and points the other way — both stay on the same line.
- LengthThe length, or norm, of v is √(2² + 1² + 2²) = 3 — Pythagoras in three dimensions. Every point on the sphere is at distance 3 from the origin.
- Data as vectorsIn a dataset, every row is a vector. Here each point is a customer described by three scaled features; similar customers sit close together.
In code, vectors are arrays and the operations are one-liners:
import numpy as np
v = np.array([2, 1, 2])
w = np.array([1, 1.5, -1])
print(v + w) # [3. 2.5 1. ]
print(1.5 * v) # [3. 1.5 3. ]
print(np.linalg.norm(v)) # 3.0Everything becomes a vector
Models do not see customers, photos or sentences — only vectors. Choosing how to turn data into vectors, called feature engineering or representation, is one of the most important decisions in any ML project.
- A table rowEach column becomes one number. Categories such as the plan are encoded as numbers too — here 1 for premium and 0 otherwise.
- An imageA grayscale image is a grid of brightness values. Laid end to end, a 3 × 3 image becomes a vector of 9 numbers; a 1-megapixel photo, a vector of a million.
- A sentenceThe simplest encoding counts how often each vocabulary word appears. Modern models use learned embeddings instead, but the result is still a vector.
- Points in spaceOnce data are vectors, each item is a point. Here every customer is placed by age and monthly visits.
- Distance means similarityThe Euclidean distance between two points measures how different they are. Customer (37, 11) is close to #1042; customer (62, 3) is far away.
- Vector
- An ordered list of numbers; geometrically an arrow or a point.
- Component
- One number in a vector, such as its value along the x-axis.
- Norm
- The length of a vector: the square root of the sum of its squared components.
- Dot product
- The sum of the products of matching components: a₁b₁ + a₂b₂ + …
- Cosine similarity
- The dot product of two vectors divided by their lengths: the cosine of the angle between them.
The dot product
The dot product multiplies two vectors into a single number. There are two ways to compute it, and they always agree:
The first form is how computers calculate it. The second explains what it means: the dot product is large and positive when the vectors point the same way, zero when they are perpendicular, and negative when they point in opposite directions. Geometrically, it is the length of b’s shadow on a, multiplied by the length of a.
Both formulas give the same number. Make the vectors perpendicular and the shadow disappears; point them apart and it flips to the other side — the dot product turns negative.
The dot product is everywhere in machine learning. A linear model’s prediction is a dot product between a weight vector and a feature vector. A neural-network layer is many dot products at once. Attention in transformers compares queries and keys with dot products.
Measuring similarity
Two measures dominate:
- Cosine similarity divides the dot product by both lengths, leaving only cos θ: 1 for the same direction, 0 for unrelated, −1 for opposite. It ignores length, which makes it ideal for text and embeddings, where length often reflects size rather than meaning.
- Euclidean distance is the length of the difference, ‖a − b‖. It compares positions, and suits data where magnitudes matter — as long as the features are on comparable scales.
Vectors describe data. The next lecture introduces matrices, which transform vectors — and are the building blocks of every model layer.
Key takeaways
- A vector is a list of numbers; geometrically, an arrow or a point in space. Any record, image or text can be encoded as one.
- Vectors add component by component and scale by multiplying every component; the length is the square root of the sum of squares.
- The dot product a·b = a₁b₁ + a₂b₂ + … equals |a||b|cos θ, so its sign says whether two vectors point the same way.
- Cosine similarity compares directions and ignores length; Euclidean distance compares positions. Both power search, recommendations and clustering.
Check your understanding
4 questions · answers explainedQ1What is the length of the vector (3, 4)?
Q2The dot product of two vectors is negative. What does that tell you?
Q3Why is cosine similarity popular for comparing documents?
Q4Two customers are described by (age, income in dollars). Why should the features be scaled before measuring distance?