CNN Fundamentals: From Convolution to Image Classification

A \(224\times224\times3\) image connected to 1,000 dense units needs 150,528,000 weights, before biases. Flattening preserves pixel values in a fixed order, but a dense layer does not build in a preference for local patterns or reuse the same detector at different positions. Convolution adds local connectivity and weight sharing. At stride 1, away from boundary effects, shifting the input shifts its feature map: this is translation equivariance. It does not guarantee that a whole classifier gives an unchanged answer after every shift, crop, or downsampling operation.

Convolution as a learned filter

A filter is a small array slid across the input, computing an elementwise product and sum at each position. Its outputs form a feature map. The vertical-edge filter below uses one grayscale channel, stride 1, no padding, and no bias. Run the Python blocks in order; later examples reuse the imports and helper functions.

import numpy as np

image = np.array([[10, 10, 10, 0, 0, 0],
                  [10, 10, 10, 0, 0, 0],
                  [10, 10, 10, 0, 0, 0],
                  [10, 10, 10, 0, 0, 0]], dtype=float)

kernel = np.array([[1, 0, -1],
                   [1, 0, -1],
                   [1, 0, -1]], dtype=float)

def conv2d_valid(x, k):
    kh, kw = k.shape
    out = np.zeros((x.shape[0] - kh + 1, x.shape[1] - kw + 1))
    for i in range(out.shape[0]):
        for j in range(out.shape[1]):
            out[i, j] = np.sum(x[i:i + kh, j:j + kw] * k)
    return out

print(conv2d_valid(image, kernel))
# [[ 0. 30. 30.  0.]
#  [ 0. 30. 30.  0.]]

For the patch beginning at column 1, each row is [10, 10, 0], whose dot product with [1, 0, −1] is 10. Three rows give 30. A constant patch gives zero because each kernel row sums to zero. Reversing the bright and dark sides reverses the sign, so this is a signed response to one edge orientation. In a CNN, backpropagation supplies gradients for the filter coefficients, and an optimizer adjusts them to reduce the training objective; it need not discover a particular edge filter or the global minimum.

The code and PyTorch Conv2d use cross-correlation: the filter is not spatially reversed. Mathematical convolution flips both spatial axes. For unconstrained learned kernels, the two parameterizations can represent the same mappings, but fixed filters, imported weights, and comparisons with signal-processing code must use a consistent convention.

The architectures these ideas came from are traced in CNN Architectures Compared: LeNet to EfficientNet.

Padding and stride

With stride 1 and dilation 1, valid convolution reduces a spatial dimension by \(f-1\), where \(f\) is the filter width. Zero padding extends the input so that output size can be preserved; border pixels can still participate in fewer windows than interior pixels, and zeros change the boundary conditions. A stride \(s\) is the spacing between filter positions. Values above 1 downsample the feature map and can make it sensitive to the alignment of a shifted input.

For an input dimension \(n\), symmetric padding \(p\) on each side, filter size \(f\), stride \(s\), and dilation \(d\), \[n_{\text{out}}=\left\lfloor\frac{n+2p-d(f-1)-1}{s}\right\rfloor+1.\] Dilation spaces the filter taps apart; \(d=1\) is the ordinary contiguous filter used below. Apply the formula separately to height and width. The padded input must be large enough for the effective filter, and dimensions and stride must be valid positive integers.

At stride 1 and dilation 1, symmetric padding preserves size when \(p=(f-1)/2\). Odd filters give an integer padding: 1 for a 3×3 and 2 for a 5×5. Even filters can preserve size with asymmetric padding; a 2×2 filter needs one extra pixel in total along each dimension. PyTorch’s padding="same" supports stride 1; conventions for “same” with larger strides differ across APIs.

def out_size(n, f, p=0, s=1, d=1):
    return (n + 2 * p - d * (f - 1) - 1) // s + 1

print(out_size(224, 3, p=1, s=1))
print(out_size(224, 3, p=1, s=2))
print(out_size(224, 7, p=3, s=2))
print(out_size(224, 11, p=0, s=4))
# 224
# 112
# 112
# 54

Convolving volumes

An RGB image has three channels. In a standard convolution with groups=1, each output filter spans all \(C_{in}\) input channels and sums their spatial responses into one map. With \(C_{out}\) filters, the output has \(C_{out}\) channels. PyTorch stores weights as \((C_{out},C_{in},f_h,f_w)\) for this case. Grouped convolutions restrict channel connections; depthwise convolution is the case where each group receives one input channel.

For square filters, groups=1, and one bias per output channel, the parameter count is \(f^2 C_{in}C_{out}+C_{out}\). A 3×3 layer mapping 64 channels to 128 has 73,856 parameters at either 56×56 or 7×7 spatial resolution. Weight count is independent of spatial size, but activation memory and computation grow with the number of output positions. With \(g\) valid groups, the weight term becomes \(f^2(C_{in}/g)C_{out}\); omit the bias term when bias=False.

def conv_params(f, c_in, c_out):
    return f * f * c_in * c_out + c_out

print(conv_params(3, 64, 128))      # 73856
print(conv_params(1, 256, 64))      # 16448
print(224 * 224 * 3 * 1000)         # 150528000  a dense first layer

The 1×1 convolution

A 1×1 convolution with stride 1 and no padding applies the same affine channel transformation at every spatial position. For channels [2, 3, 4], weights [1, 0, −1], and zero bias, one output channel is 2 − 4 = −2. Other filters produce other channel mixtures. It can change channel count without mixing neighboring positions; a stride above 1 can also downsample. At matching channel counts and output size, its weight count is one ninth that of a 3×3 convolution.

A bottleneck can reduce 256 channels to 64, apply a 3×3 convolution, and expand back to 256. Counting biases in all three layers gives 70,016 parameters, compared with 590,080 for a direct 3×3 layer. This resembles a ResNet bottleneck’s channel reduction, but the count omits normalization and any shortcut projection. The two designs have different computations and representational constraints; the ratio is not a guarantee of equal accuracy or an 8.4× speedup. MobileNet variants use related pointwise operations with different depthwise or expansion patterns.

direct = conv_params(3, 256, 256)
bottleneck = (conv_params(1, 256, 64)
              + conv_params(3, 64, 64)
              + conv_params(1, 64, 256))
print(direct, bottleneck, round(direct / bottleneck, 1))
# 590080 70016 8.4

Pooling

Max pooling selects the largest value in each window. A 2×2 patch [[1, 3], [2, 4]] becomes 4; average pooling would give 2.5. With stride 2 and no padding, even spatial dimensions halve and odd ones round down under the default floor rule. Pooling has no learned weights. Some shifts leave a pooled maximum unchanged while others move it across a window boundary, so pooling does not guarantee translation invariance. Reduced spatial size can lower downstream cost for otherwise comparable layers.

Strided convolutions provide learned downsampling, while pooling remains useful in many architectures. Global average pooling is a common classifier design, not a universal one: it averages each channel across all spatial positions to produce a fixed-length channel vector. This removes spatial size from the head’s input shape, but changing resolution can still change feature values, predictions, and total computation. It also discards explicit spatial layout from the pooled representation.

Assembling a classifier

The classifier below uses repeated convolution, BatchNorm, and ReLU blocks, followed by downsampling. ReLU supplies nonlinear transformations; without nonlinear operations, a composition of affine layers would remain affine. PyTorch inputs here have shape (N, C, H, W): batch, channels, height, width. For 32×32 images, the three blocks produce (N, 32, 16, 16), (N, 64, 8, 8), and (N, 128, 4, 4). Global pooling yields (N, 128, 1, 1), and flattening from axis 1 gives (N, 128) for the linear head.

import torch
import torch.nn as nn

def block(c_in, c_out):
    return nn.Sequential(
        nn.Conv2d(c_in, c_out, kernel_size=3, padding=1, bias=False),
        nn.BatchNorm2d(c_out),
        nn.ReLU(inplace=True),
        nn.Conv2d(c_out, c_out, kernel_size=3, padding=1, bias=False),
        nn.BatchNorm2d(c_out),
        nn.ReLU(inplace=True),
        nn.MaxPool2d(2),
    )

class SmallCNN(nn.Module):
    def __init__(self, num_classes=10):
        super().__init__()
        self.features = nn.Sequential(block(3, 32), block(32, 64), block(64, 128))
        self.pool = nn.AdaptiveAvgPool2d(1)      # global average pooling
        self.head = nn.Linear(128, num_classes)

    def forward(self, x):
        x = self.features(x)
        x = self.pool(x).flatten(1)
        return self.head(x)

torch.manual_seed(0)
model = SmallCNN()
x = torch.randn(2, 3, 32, 32)
print(model(x).shape)
print(sum(p.numel() for p in model.parameters()))
# torch.Size([2, 10])
# 288746

For this feature extractor, positive batch size and both spatial dimensions at least 8 survive three 2×2 pooling stages. Global pooling then makes the head compatible with different rectangular resolutions. A batch still needs compatible tensor shapes or an explicit padding/batching strategy, and shape compatibility does not guarantee good performance on a new resolution.

The head returns ten logits per image, one unnormalized score for each class. Cross-entropy takes those logits directly and integer target indices 0–9; do not apply softmax before this loss. Backpropagation carries the classification loss through pooling and the feature blocks into the learned filters. The synthetic labels below only demonstrate one update and prediction, not recognition quality.

import torch.nn.functional as F

model.train()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
labels = torch.tensor([1, 3], dtype=torch.long)
old_weights = model.features[0][0].weight.detach().clone()
optimizer.zero_grad(set_to_none=True)
logits = model(x)
loss = F.cross_entropy(logits, labels)
loss.backward()
optimizer.step()
print("finite loss", bool(torch.isfinite(loss).item()))
print("first filter updated", not torch.equal(old_weights, model.features[0][0].weight))

model.eval()
with torch.no_grad():
    logits = model(x)
    predicted_classes = logits.argmax(dim=1)
    probabilities = logits.softmax(dim=1)
print("prediction shape", tuple(predicted_classes.shape))
print("probabilities sum to one", torch.allclose(probabilities.sum(1), torch.ones(2)))
# finite loss True
# first filter updated True
# prediction shape (2,)
# probabilities sum to one True

At inference, argmax selects the class with the largest score. Softmax converts the scores into values that sum to one, but does not ensure calibrated confidence. A real classifier needs representative labeled images, fixed class-index meanings, an appropriate preprocessing pipeline, and validation on held-out data. The full data-loader and epoch loop is developed in Building a PyTorch Training Loop.

Receptive field

The nominal receptive field describes the input region that can reach one feature-map unit through the spatial computation graph; the actual influence depends on weights and activations. At stride 1 and dilation 1, two 3×3 convolutions reach 5×5 and three reach 7×7. With constant channel width \(C\) throughout and biases excluded, their weight counts are \(18C^2\) versus \(25C^2\), and \(27C^2\) versus \(49C^2\), respectively. Different intermediate widths change these comparisons. ReLUs between the small convolutions introduce nonlinearities, so the stack is not simply an equivalent larger filter.

Track the receptive-field width \(r\) and spacing \(j\) between neighboring outputs measured in input pixels. Start with \(r=1,j=1\). A layer of effective kernel width \(f\) and stride \(s\) updates them as \(r\leftarrow r+(f-1)j\), then \(j\leftarrow sj\). Two 3×3 convolutions followed by 2×2 pooling give \(r=6,j=2\); applying the same block twice more gives \(r=16,j=4\) and \(r=36,j=8\). Padding affects where these regions lie and how much is real input. This calculation treats normalization as fixed at inference; training-mode BatchNorm also couples spatial locations and batch examples through its statistics.

The final classifier aggregates multiple feature-map positions through global pooling, so its input coverage differs from that of one local feature unit. Useful context depends on the task: local texture can sometimes classify an object, while other decisions need relationships across a wide region. A receptive field larger than the object is neither a universal requirement nor a guarantee of recognition.

The encoder-decoder with skip connections is built in Semantic Segmentation and U-Net Explained.

Implementation checks

• PyTorch Conv2d uses \((N,C,H,W)\). Many image arrays use \((H,W,C)\); reorder axes with a permutation, not a reshape, and inspect channel values as well as dimensions.
• Inspect stage shapes on a small batch. Also check dtype, value range, channel order, and label mapping; valid shapes alone do not establish correct inputs.
• A convolutional bias is usually redundant when followed immediately by trainable BatchNorm: batch centering removes the constant during training, and the affine BatchNorm supplies an offset. Removing a bias from an already trained model without adjusting its state is a different operation.
• For pretrained weights, follow the specified resizing, color ordering, scaling, and normalization. For a new training pipeline, estimate any learned preprocessing statistics on training data and apply that same protocol during evaluation.
• Inspect augmented images and labels together. Spatial transforms must also update boxes, masks, or coordinates when those targets are present. For classification, verify that the transformation preserves the class; mismatches can corrupt supervision even though a loss still computes.

What CNN layers actually represent, and how to use that, is shown in Neural Style Transfer: Feature Visualization, Gram Matrices, and Optimization.

Exercises

1. Output size arithmetic. Compute the output spatial size for each: \(224\) input with \(f=3,p=1,s=1\); \(224\) with \(f=7,p=3,s=2\); \(224\) with \(f=3,p=0,s=2\); \(32\) with \(f=5,p=2,s=1\). Which configurations preserve the input size? For stride 1, dilation 1, and symmetric padding, what condition preserves size?

You should get: four sizes, two of which equal the input, and one condition relating \(p\) and \(f\).

def out_size(n, f, p=0, s=1):
    return (n + 2*p - f) // s + 1
print(out_size(224, 3, 1, 1))   # 224
print(out_size(224, 7, 3, 2))   # 112
print(out_size(224, 3, 0, 2))   # 111
print(out_size(32, 5, 2, 1))    # 32

2. The bottleneck saving. Compare the parameter count of a direct \(3\times3\) convolution on 256 channels against a \(1\times1\) reduction to 64, a \(3\times3\), and a \(1\times1\) expansion back to 256. Report the ratio, then recompute with a reduction to 128 instead.

You should get: a ratio above 8 for the narrow bottleneck and a much smaller one for the wide version.

def conv_params(f, c_in, c_out): return f*f*c_in*c_out + c_out
direct = conv_params(3, 256, 256)
for mid in (64, 128):
    bn = conv_params(1,256,mid) + conv_params(3,mid,mid) + conv_params(1,mid,256)
    print(mid, direct, bn, round(direct/bn, 2))
# 64 590080 70016 8.43
# 128 590080 213504 2.76

3. Receptive field and depth. Compute the receptive field of a stack of \(k\) 3×3 convolutions with stride 1 and dilation 1. Assume every layer has 64 input and output channels, and count weights only, excluding biases. Compare against a single convolution with the same receptive field for \(k=2\) and \(k=3\).

You should get: equal receptive fields with fewer parameters in the stacked version, plus something the single layer lacks.

c = 64
for k in (2, 3, 4):
    rf = 2*k + 1
    print(k, rf, 9*k*c*c, rf*rf*c*c)
# 2 5 73728 102400
# 3 7 110592 200704
# 4 9 147456 331776

References

PyTorch Conv2d specifies channel groups, padding, and output shapes; CrossEntropyLoss explains logits and target formats. Simonyan and Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition, develops the VGG small-filter architecture.