1. Courses
  2. Computer Vision
  3. Lecture 4
Lecture 4 of 1418 minInteractiveFree lecture

Training image classifiers

Good image classifiers come from good data practice as much as good architectures. Multiply your data with augmentation, read a confusion matrix to find what the model gets wrong, and use transfer learning to reach high accuracy with only hundreds of labeled images.

In this lecture you will

  • Explain how data augmentation improves generalization
  • Read a confusion matrix and compute per-class precision and recall
  • Apply transfer learning — freezing a pretrained backbone and fine-tuning it
  • Plan the data work for a real image-classification project

You now know what a CNN is. Turning one into a classifier that works reliably on real photos is mostly about data and evaluation: how to make the most of a limited set of labeled images, how to see what the model gets wrong, and how to borrow knowledge from models trained on millions of images.

Augmentation: more data for free

A model should recognize a car whether it faces left or right, in morning or evening light, centered or slightly cropped. Rather than collecting photos of every variation, we can generate them: every time an image is used in training, apply a random, label-preserving transformation.

InteractiveData augmentation playground
Loading visual…

Each training epoch sees a different random version of every photo. The model cannot memorize exact pixels, so it has to learn what stays the same: the objects.

Flip
Mirror the image horizontally. Safe for most natural scenes, wrong for text and anything with meaningful left–right orientation.
Rotate / crop / zoom
Small geometric changes teach the model that position and scale do not change the class.
Color and lighting jitter
Random brightness, contrast and saturation changes teach robustness to cameras and lighting.
Cut-out
Blank out a random patch so the model cannot rely on one small detail and must use the whole object.
Test-time augmentation
Predict on several augmented versions of a test image and average the results — a cheap accuracy boost.

Reading a confusion matrix

A single accuracy number hides a lot. A confusion matrix counts, for every true class (rows), how often each class was predicted (columns). Click any cell to see what it means.

InteractiveWhere does the classifier go wrong?Click a cell
Loading visual…

Most errors cluster in two pairs: cats and dogs, cars and trucks. That tells you where to collect more examples or add features — something overall accuracy could never show.

From the matrix we can compute, for each class:

precision=correct predictions of the classall predictions of the classrecall=correct predictions of the classall real examples of the class\text{precision} = \frac{\text{correct predictions of the class}}{\text{all predictions of the class}} \qquad \text{recall} = \frac{\text{correct predictions of the class}}{\text{all real examples of the class}}

Which matters more depends on the cost of each mistake. A defect detector that misses cracks (low recall) is dangerous; one that flags too many good parts (low precision) is merely expensive.

Transfer learning: stand on the shoulders of ImageNet

Training a CNN from scratch needs hundreds of thousands of labeled images. Few companies have that for their own problem — but the early and middle layers of any vision model learn similar things: edges, textures, shapes, parts. Transfer learning reuses them.

AnimationTransfer learning, step by step
Loading animation…
  1. Start from a pretrained backboneA network trained on ImageNet — 1.2 million photos in 1,000 classes — has learned general visual features in its blocks.
  2. Replace the headThrow away the 1,000-class output layer and attach a small new head for your own classes: here, OK, scratch and dent.
  3. Freeze the backbone, train the headLock the backbone’s weights and train only the new head. It takes minutes, and the general features are reused unchanged.
  4. Fine-tune the top blocksUnfreeze the last blocks and continue with a small learning rate, so the high-level features adapt to your images without forgetting what they know.
  5. Better results from less dataWith the same 600 labeled photos, a fine-tuned pretrained model typically far outperforms one trained from scratch.
import torch
from torch import nn
from torchvision import models

model = models.resnet50(weights="IMAGENET1K_V2")    # pretrained backbone
for p in model.parameters():
    p.requires_grad = False                         # 1. freeze everything

model.fc = nn.Linear(model.fc.in_features, 3)       # 2. new head: OK / scratch / dent
optimizer = torch.optim.AdamW(model.fc.parameters(), lr=1e-3)
# ... train the head for a few epochs ...

for p in model.layer4.parameters():                 # 3. unfreeze the top block
    p.requires_grad = True
optimizer = torch.optim.AdamW(
    [{"params": model.fc.parameters(), "lr": 1e-4},
     {"params": model.layer4.parameters(), "lr": 1e-5}]   # small learning rates
)

Classification answers what is in an image. Many applications also need where — counting products on a shelf, finding every pedestrian, locating a defect. That is object detection, the subject of the next lecture.

Key takeaways

  • Augmentation creates new, realistic training images by flipping, rotating, cropping and changing lighting, so the model learns what does not matter.
  • A confusion matrix shows exactly which classes are mistaken for which — far more useful than accuracy alone.
  • Transfer learning reuses a backbone pretrained on millions of images; fine-tuning it needs far less data and compute.
  • Most accuracy gains in practice come from better, more representative data and careful evaluation.

Check your understanding

4 questions · answers explained

Q1Why should horizontal flips NOT be used when training a model to read text in images?

An augmentation must preserve the label. Flipping a cat still gives a cat, but a flipped “b” reads as “d”. Choose augmentations that match the variation you expect in real data.

Q2In a confusion matrix, where do you find the correct predictions?

Rows are true classes and columns are predictions. Cells on the diagonal count correct predictions; every off-diagonal cell is a specific kind of mistake.

Q3Precision for “dog” is 83% and recall is 80%. What does the recall tell you?

Recall is the fraction of actual dogs that were recognized as dogs. Precision answers the other question — of everything called “dog”, how much really was a dog.

Q4You have 500 labeled images of your products. What is usually the best starting point?

A pretrained backbone already knows edges, textures and object parts. Adapting it needs only a small dataset, trains quickly and usually generalizes far better than training from scratch.