Convolutional neural networks
A convolutional neural network stacks learned filters, pooling and a small classifier. Fly through one in 3D, see the feature maps its first layer produces, and understand why each layer sees more of the image than the last.
In this lecture you will
- Describe the layers of a CNN — convolution, activation, pooling and dense layers
- Explain what feature maps are and what early filters learn to detect
- Explain how pooling and depth grow the receptive field
- Estimate the parameters and output shape of a convolutional layer
In the last lecture we designed kernels by hand: a vertical-edge detector, a blur, an outline. That works for simple effects, but nobody could hand-design the thousands of filters needed to recognize a cat, a tumour or a scratch on a phone screen. The breakthrough of convolutional neural networks (CNNs) is to treat the kernel weights as parameters and learn them from labeled examples with backpropagation.
What a CNN layer produces
A convolutional layer applies many filters to the same input. Each filter produces one feature map: a grid saying where its pattern appears. The maps are stacked, so the output of a layer is itself an image with many channels.
Each map is the whole image seen through one filter. Trained CNNs learn filters remarkably like these in their first layer: edges at several orientations and color contrasts.
Switch off ReLU and you will see negative responses in coral: the diagonal filters, for example, respond positively to edges in one direction and negatively to the opposite. ReLU keeps only the positive evidence, which makes the next layer’s job simpler.
Pooling and the growing field of view
Between convolutions, CNNs usually downsample. Max-pooling keeps the strongest response in each small block, making the maps smaller and the network less sensitive to exactly where a feature appeared.
- Split the map into blocksA 4 × 4 feature map is divided into four non-overlapping 2 × 2 blocks, shown in different colors.
- Keep the maximumMax-pooling keeps the largest value of each block. Sixteen numbers become four, and the strongest evidence survives.
- Or keep the averageAverage-pooling keeps each block’s mean instead. It is smoother; inside CNNs max-pooling is more common, while a global average often sits right before the classifier.
- The receptive field growsA layer-2 unit sees a 3 × 3 patch of layer 1, and those units together see 5 × 5 input pixels. Every layer and every pooling step widens the view.
That growing receptive field is why depth matters. The first layer sees only 3 × 3 pixels — enough for an edge. A few layers later a unit sees a textured patch; deeper still, whole wheels, windows and faces.
Fly through a CNN
The network below is built from real feature maps of the course scene. Scroll to move through it layer by layer.
- 1The input image
Three channels of pixels enter the network: here a 96 × 66 color image.
- 2Convolution layer 1
Six learned filters slide over the image. Each produces a feature map, and ReLU clears the negative responses. Edges and color contrasts light up.
- 3Pooling
Max-pooling keeps the strongest response in every 2 × 2 block, halving each map. The network keeps what was found, not exactly where.
- 4Convolution layer 2
The next layer combines all the previous maps. Each of its units sees a larger patch of the original image — the amber cone shows one unit’s receptive field.
- 5From features to classes
After more pooling, the maps are flattened into a vector. Fully connected layers turn them into one score per class, and softmax gives probabilities.
- 6Everything is learned
Every filter weight, in every layer, is adjusted by backpropagation so that the correct class gets a higher probability. No filter is designed by hand.
- Convolutional layer
- A layer of learned filters. Each filter spans all input channels and produces one output channel.
- Feature map
- The output of one filter: where, and how strongly, its pattern appears.
- Pooling
- Downsampling that summarizes each small block, usually by its maximum.
- Receptive field
- The region of the input image that can influence one unit. It grows with every layer.
- Backbone
- The convolutional feature extractor of a network, reused for many tasks. The classifier on top is the head.
Counting parameters and shapes
A filter spans all input channels. A 3 × 3 convolution from 16 channels to 32 channels therefore has weights plus 32 biases — independent of the image size. A classic small CNN for 32 × 32 images:
import torch
from torch import nn
model = nn.Sequential(
nn.Conv2d(3, 32, kernel_size=3, padding=1), nn.ReLU(), # 32×32×32
nn.MaxPool2d(2), # 16×16×32
nn.Conv2d(32, 64, kernel_size=3, padding=1), nn.ReLU(), # 16×16×64
nn.MaxPool2d(2), # 8×8×64
nn.Conv2d(64, 128, kernel_size=3, padding=1), nn.ReLU(), # 8×8×128
nn.AdaptiveAvgPool2d(1), # 1×1×128
nn.Flatten(),
nn.Linear(128, 10), # one score per class
)
print(model(torch.randn(1, 3, 32, 32)).shape) # torch.Size([1, 10])Key takeaways
- A CNN learns its filters from data instead of using hand-designed kernels.
- Each convolutional layer produces a stack of feature maps, one per filter; ReLU keeps positive responses.
- Pooling (or strided convolution) shrinks the maps, so later layers work on smaller grids but see larger regions.
- Early layers detect edges and colors, middle layers textures and parts, late layers whole objects; a dense head turns them into class scores.
Check your understanding
4 questions · answers explainedQ1A convolutional layer has 32 filters of size 3 × 3 and receives an input with 16 channels. How many weights does it have (ignoring biases)?
Q2What does a 2 × 2 max-pooling layer do to a 64 × 64 × 32 input?
Q3Why can a unit in a deep layer respond to a whole object, while first-layer units only respond to edges?
Q4What typically sits at the end of a CNN image classifier?