Convolutional Neural Networks

The Problem with Treating Images as Flat Vectors

A standard neural network takes a flat vector of numbers as input. If you feed it a 32×32 image, it becomes a vector of 1,024 pixel values (or 3,072 for colour). The network has no idea that pixel [5,6] is adjacent to pixel [5,7] — that spatial relationship is completely lost when you flatten the image.

For simple digit recognition on clean, centred images, this limitation is workable but costly: the network must learn the concept of "edge" or "curve" independently for every possible position in the image. The same horizontal stroke at the top of a "7" and at the bottom of a "1" would need to be learned as two completely different features. This is why ANN models on SVHN plateau around 77%.

The key insight CNNs exploit Visual features are spatially local and translation-invariant. An edge is an edge whether it appears in the top-left or bottom-right of an image. A CNN learns a feature detector once and applies it everywhere — a fundamental efficiency that plain ANNs lack.

The Convolution Operation

A convolutional filter (also called a kernel) is a small grid of learnable weights — typically 3×3 or 5×5. To apply it to an image, you slide it across the image one step at a time and compute the dot product at each position. The output is a feature map: a grid showing where in the image the filter's pattern was detected, and how strongly.

Example: 3×3 edge-detection filter applied to a 6×6 image

Filter (detects vertical edges):      Image patch (excerpt):
  -1  0  +1                              10  10  200
  -1  0  +1                              10  10  200
  -1  0  +1                              10  10  200

Dot product: (-1×10)+(0×10)+(+1×200) × 3 rows = +570
→ Large positive value: strong vertical edge detected here

At a flat region (e.g., 10 10 10 / 10 10 10 / 10 10 10):
Dot product = 0 → no edge detected

The network learns these filters through backpropagation — not by hand. Early layers learn simple features (edges, corners, blobs). Later layers combine these to detect more complex features (curves, loops, digit-like shapes).

Multiple Filters → Multiple Feature Maps

A single convolutional layer typically applies 32, 64, or 128 different filters simultaneously. Each filter learns to detect a different type of feature. The output of a convolutional layer is a stack of feature maps — one per filter. A layer with 64 filters on a 32×32 image produces a 32×32×64 output volume.

MaxPooling — Downsampling with Intent

After a convolutional layer, the feature maps can be large. MaxPooling reduces their spatial dimensions by dividing the map into non-overlapping windows (typically 2×2) and keeping only the maximum value from each window.

MaxPooling 2×2 on a 4×4 feature map:

Input:          Output:
1  3  2  4      3  4
5  6  1  2  →   6  5
2  1  3  2      4  3
4  2  1  3

Each 2×2 block → one output value (the maximum)

Two effects: it halves the spatial dimensions (reducing computation), and it introduces a small amount of translation invariance — a feature detected slightly off-centre still produces the same pooled output. Crucially, it keeps the most "activated" detection signal from each region.

The Full CNN Architecture

A typical CNN stacks convolutional + pooling blocks, then flattens and connects to fully-connected layers for the final classification decision:

Input

Image tensor

Shape: (height, width, channels). For SVHN: (32, 32, 3) — 32×32 pixels, RGB colour. Total: 3,072 values per image.

Feature extraction (repeating blocks)

Conv2D → BatchNorm → LeakyReLU → MaxPooling

Each block applies learnable filters, normalises activations, introduces non-linearity, then downsamples. Early blocks detect simple features; later blocks detect complex compositions. Filters increase per block (e.g., 32 → 64 → 128) to capture more abstract features.

Transition

Flatten

Converts the 3D feature volume (height × width × filters) into a 1D vector. This is where spatial structure is finally collapsed — but by now, the network has already extracted all the spatial patterns it needs.

Classification head

Dense → Dropout → Dense → Softmax

Fully-connected layers combine the extracted features to make a final classification decision. Dropout regularises. Softmax outputs class probabilities.

Parameter efficiency A Dense layer connecting 1,024 inputs to 512 neurons requires 1,024 × 512 = 524,288 parameters. A Conv2D layer with 64 3×3 filters has only 64 × 3 × 3 × input_channels parameters — shared across the entire image. Convolutional weight sharing is what makes deep image models trainable.

Why CNNs Outperform ANNs on Images

Local connectivity: Each neuron looks at a small patch of the image, not the entire input. Spatial structure is preserved.
Weight sharing: The same filter is used across the whole image — a feature detector learned once applies everywhere. Far fewer parameters than equivalent dense layers.
Hierarchical feature learning: Low-level (edges) → mid-level (shapes) → high-level (digit structure). This hierarchy mirrors how biological visual systems process information.
Translation invariance (partial): MaxPooling makes CNNs somewhat robust to small spatial shifts — a digit shifted a few pixels in any direction is still correctly classified.

Interactive: Convolution Walkthrough

Click Step to advance the filter one position at a time. The highlighted patch on the input shows exactly which pixels are being processed. The dot product with the filter weights produces a single value in the feature map — that is the entire convolution operation.

Sobel Vertical Edge Detector · 3×3 filter on 6×6 image

Input Image (6×6)

⊗

Filter Weights (3×3)

→

Feature Map (4×4)

Position 0/16 — Click Step or Auto to begin

In the SVHN Project

CNN1 used two convolutional blocks (32→64 filters, 3×3 kernels) followed by dense layers, achieving 87.30% accuracy — a 10-point jump over the best ANN. CNN2 added a third convolutional block (128 filters), deeper dense layers, and more aggressive regularisation, reaching 92.22%. The spatial feature maps learned by the CNN captured digit-specific patterns (loop closures, stroke angles, curve directions) that the flat ANN simply couldn't represent efficiently. See Chapter 6 — CNN Architecture →

← Overfitting Next: Model Evaluation →