Understanding convolution, padding, stride, pooling, and fully connected layers gives you the basic components of a convolutional neural network. The next challenge is learning how to combine those components into an effective architecture.

There is no single formula that determines:

• How many convolutional layers to use
• Where to reduce spatial resolution
• How quickly to increase channel counts
• Whether to use pooling or strided convolution
• How to train a very deep network
• How to balance accuracy and computational cost

Many of the most useful design principles were discovered by studying architectures that succeeded on large and difficult computer vision tasks.

From Building Blocks to Complete Architectures

A basic CNN may contain:\[ \text{convolution} \rightarrow \text{activation} \rightarrow \text{pooling} \rightarrow \text{fully connected layer} \]

But real architectural design requires more detailed decisions.

For every stage, a designer must select:

• Filter dimensions
• Number of filters
• Stride
• Padding
• Activation function
• Normalization method
• Downsampling strategy
• Connection pattern
• Classification head

These choices interact. A configuration that works well in isolation may not work well when placed inside a deep network.

For example, aggressive downsampling reduces computational cost but may destroy useful spatial information. Increasing the channel count improves representation capacity but also increases memory and computation.

Studying successful models shows how researchers balanced these tradeoffs.

Learning Architecture Design Through Case Studies

One effective way to learn software design is to read well-structured programs written by experienced developers.

CNN architecture can be learned similarly:

1. Examine a successful network.
2. Follow the dimensions through every layer.
3. Identify repeated design patterns.
4. Understand the problem each architectural idea solves.
5. Determine which ideas transfer to other tasks.

A model architecture is more than a list of layers. It represents a collection of design decisions shaped by:

• Available hardware
• Dataset size
• Optimization limitations
• Accuracy requirements
• Memory constraints
• Previous experimental findings

Historical architectures are especially useful because they introduce important ideas one step at a time.

Why Successful Architectures Often Transfer

An architecture developed for one visual task can often work well on another.

Suppose a network learns to recognize:

• Animals
• Vehicles
• People
• Buildings
• Everyday objects

Its early and intermediate layers may learn broadly useful features:\[ \text{edges} \rightarrow \text{textures} \rightarrow \text{shapes} \rightarrow \text{object parts} \]

These visual features can also support applications such as:

• Autonomous perception
• Medical-image analysis
• Product recognition
• Satellite-image interpretation
• Industrial inspection
• Facial analysis

This transfer can happen at two levels.

Architecture transfer

Reuse the structural design but initialize new parameters:\[ \text{established architecture} + \text{new random weights} \]

Parameter transfer

Reuse both the architecture and pretrained parameters:\[ \text{pretrained network} \rightarrow \text{adaptation to a new task} \]

The second approach is commonly known as transfer learning.

Architecture Transfer vs. Transfer Learning

These ideas are related but distinct.

Method	Reused architecture	Reused parameters
New model designed from scratch	No	No
Architecture transfer	Yes	No
Transfer learning	Yes	Yes

An architecture may remain valuable even when its original weights cannot be reused.

For example, a network designed for natural-image recognition may provide a good structural starting point for another image task, even if the new domain requires training all parameters again.

What to Look for in an Architecture

When studying a CNN, do not focus only on memorizing layer counts. Instead, ask what pattern the architecture follows.

Spatial progression

Track the height and width:\[ n_H^{(0)},n_W^{(0)} \rightarrow n_H^{(1)},n_W^{(1)} \rightarrow \cdots \]

Ask:

• When does downsampling occur?
• Does it happen through pooling or convolution?
• How quickly is spatial resolution reduced?

Channel progression

Track the number of channels:\[ n_C^{(0)} \rightarrow n_C^{(1)} \rightarrow \cdots \]

Many networks increase channels as spatial dimensions decrease.

Repeated blocks

Look for reusable patterns such as:\[ \text{CONV} \rightarrow \text{CONV} \rightarrow \text{POOL} \]

or:\[ \text{CONV} \rightarrow \text{NORMALIZATION} \rightarrow \text{ACTIVATION} \]

or:\[ \text{MAIN PATH} + \text{SKIP PATH} \]

Repeated blocks often reveal the architecture’s central idea.

Receptive-field growth

Determine how quickly deeper units obtain access to broader image regions.

Small filters stacked deeply can produce large effective receptive fields while adding nonlinear transformations between them.

Parameter distribution

Identify where most parameters are located:

• Convolutional feature extractor
• Fully connected classifier
• Projection layers
• Bottleneck blocks

Older networks often concentrate parameters in dense layers. Later architectures frequently reduce or eliminate these large classifiers.

Computational bottlenecks

Parameter count alone does not determine computational cost.

A convolution with relatively few weights can still be expensive when applied across a large feature map. Therefore, examine:

• Spatial dimensions
• Channel counts
• Filter sizes
• Number of operations
• Activation memory

Optimization strategy

Very deep architectures need mechanisms that make optimization stable.

Look for:

• Residual connections
• Normalization
• Appropriate initialization
• Activation placement
• Auxiliary outputs
• Bottleneck structures

Main Architecture Families

Several historically influential network families reveal different stages in CNN development.

LeNet-5

LeNet-5 established an early and enduring CNN pattern:\[ \text{CONV} \rightarrow \text{SUBSAMPLING} \rightarrow \text{CONV} \rightarrow \text{SUBSAMPLING} \rightarrow \text{CLASSIFIER} \]

It was designed for character and document recognition and used a relatively small number of parameters.

The architecture’s roots extend to convolutional work from the late 1980s, while the well-known LeNet-5 description appeared in the 1998 document-recognition paper. Original LeNet-5 paper

Important ideas include:

• Local receptive fields
• Shared convolutional weights
• Gradual spatial reduction
• Increasing feature channels
• End-to-end training

LeNet-5 provides a clear starting point for understanding complete convolutional systems.

AlexNet

AlexNet scaled convolutional learning to a large image-recognition problem.

Its major characteristics included:

• Five convolutional layers
• Three fully connected layers
• ReLU activations
• GPU-based training
• Max pooling
• Dropout
• Data augmentation
• Approximately 60 million parameters

AlexNet showed that a large CNN trained on a large labeled dataset could achieve dramatically improved visual-recognition performance. Original AlexNet paper

Its architecture was less uniform than later networks, but its historical impact was enormous.

VGG Networks

VGG simplified CNN design using repeated small convolutions:\[ 3\times3,\qquad s=1,\qquad \text{same padding} \]

Downsampling was performed using:\[ 2\times2\text{ max pooling},\qquad s=2 \]

The architecture followed a regular pattern:

• Preserve spatial dimensions within a block.
• Halve spatial dimensions after pooling.
• Increase channels as resolution decreases.
• Stack multiple small filters to create larger effective receptive fields.

VGG-16 contains 16 trainable layers, while VGG-19 extends the design to 19. The original study demonstrated that greater depth using small filters could substantially improve large-scale recognition. Original VGG paper

Residual Networks

As networks become deeper, they can become more difficult to optimize. Simply adding layers does not guarantee lower training error.

Residual networks address this with shortcut connections.

A residual block computes:\[ A^{(l+2)} = g\left( F\left(A^{(l)}\right) + A^{(l)} \right) \]

Instead of forcing the stacked layers to learn a complete transformation, the block learns a residual function:\[ F(x)=H(x)-x \]

so that:\[ H(x)=F(x)+x \]

The shortcut provides a direct path for information and gradients.

Residual connections enabled successful training of much deeper networks, including a widely recognized 152-layer model.

Why Residual Connections Matter

If the residual branch learns:\[ F(x)\approx0 \]

then the block behaves approximately like an identity mapping:\[ H(x)\approx x \]

This makes it easier for additional layers to preserve an already useful representation rather than degrading it.

Residual connections became one of the most influential architectural ideas in deep learning and are now used well beyond computer vision.

Inception Networks

Inception architectures address a different question:

What filter size or operation should a layer use?

Instead of choosing only one, an Inception module may perform several operations in parallel:\[ 1\times1\text{ convolution} \]\[ 3\times3\text{ convolution} \]\[ 5\times5\text{ convolution} \]\[ \text{pooling} \]

The results are concatenated along the channel dimension.

Conceptually:\[ X \rightarrow \begin{cases} \text{branch 1}\\ \text{branch 2}\\ \text{branch 3}\\ \text{branch 4} \end{cases} \rightarrow \text{concatenated output} \]

This lets the network process visual information at multiple spatial scales.

Inception also uses \(1\times1\) convolutions as bottlenecks to reduce channel dimensions and control computational cost.

Different Architectures Solve Different Problems

Architecture	Central architectural lesson
LeNet-5	Combine convolution, subsampling, and classification
AlexNet	Scale CNNs with data, GPU computation, and ReLU
VGG	Build deep networks from simple, uniform blocks
ResNet	Use shortcut connections to optimize very deep networks
Inception	Process multiple scales efficiently using parallel branches

These models should not be studied merely as historical lists of layers. Each one addresses a particular design challenge.

Ideas That Generalized Beyond Vision

Several CNN innovations later appeared in other areas.

Residual connections

Residual pathways are now used in:

• Language models
• Speech systems
• Generative models
• Graph neural networks
• Scientific machine learning

Multiscale processing

Parallel or hierarchical representations are useful for:

• Audio
• Time series
• Language
• Sensor fusion

Bottleneck layers

Dimensionality-reduction projections help control computation in many deep architectures.

Reusable blocks

Modern networks are commonly built from repeated modules rather than independently designed layers.

This cross-domain influence is one reason classic vision architectures remain worth studying.

How to Read an Architecture Paper

A systematic reading strategy makes technical papers easier to understand.

Start with the architecture diagram

Identify:

• Input dimensions
• Major stages
• Downsampling locations
• Output dimensions
• Repeated blocks

Reconstruct tensor shapes

For every convolution:\[ n_{\text{out}} = \left\lfloor \frac{n+2p-f}{s} \right\rfloor+1 \]

For pooling, apply the corresponding spatial formula.

Check that the reported dimensions are internally consistent.

Count parameters selectively

For a standard convolution:\[ \text{parameters} = n_C^{\text{out}} \left( f_Hf_Wn_C^{\text{in}}+1 \right) \]

For a dense layer:\[ \text{parameters} = n_{\text{in}}n_{\text{out}} + n_{\text{out}} \]

This reveals where model capacity is concentrated.

Separate enduring ideas from historical constraints

Some design choices were shaped by older hardware.

Examples include:

• Splitting AlexNet across two GPUs
• Partial connectivity in LeNet-5
• Very large fully connected classifiers
• Local response normalization

Understand why the choice was made, but do not assume it remains a default.

Find the main contribution

Ask:

• What problem did the existing architectures have?
• What new mechanism was introduced?
• Why should it solve the problem?
• What evidence supports the claim?
• Which parts are essential and which are implementation details?

This prevents the architecture from becoming a collection of disconnected numbers.

Use Paper Results Carefully

A model’s published performance depends on more than its architecture.

It may also depend on:

• Data preprocessing
• Data augmentation
• Training duration
• Initialization
• Optimization
• Learning-rate schedules
• Ensembling
• Evaluation procedures

Reproducing only the layer structure may not reproduce the original result.

Why Not Invent Every Architecture from Scratch?

The architecture search space is enormous.

Even a small network requires choices for:

• Depth
• Width
• Filter sizes
• Strides
• Padding
• Pooling
• Activation functions
• Normalization
• Skip connections
• Regularization

Established architectures encode many design decisions that have already survived extensive experimentation.

A practical workflow is:

1. Find a successful architecture for a related task.
2. Understand its input and output assumptions.
3. Adapt the classification head.
4. Adjust resolution or channel counts if necessary.
5. Reuse pretrained parameters when appropriate.
6. Evaluate and modify only where evidence suggests a need.

Architecture Names Are Not Always Exact Specifications

The same architecture name may refer to slightly different implementations.

Differences can include:

• Input resolution
• Padding convention
• Layer ordering
• Bias usage
• Normalization
• Classification head
• Framework defaults

For example, AlexNet descriptions may use either 224 or 227 as the input width, depending on cropping and padding conventions.

When exact reproducibility matters, consult the specific implementation and verify every tensor shape.

Historical Importance vs. Current Use

Classic networks remain educationally valuable even when newer architectures are more efficient.

Architecture	Historical value	Current practical limitation
LeNet-5	Clear introduction to CNN structure	Too small for complex modern vision
AlexNet	Demonstrated large-scale CNN success	Computationally inefficient by current standards
VGG-16	Elegant and uniform feature extractor	Very large parameter and memory requirements
ResNet	Established highly reusable residual blocks	Many specialized variants now exist
Inception	Demonstrated efficient multiscale processing	More structurally complex than uniform networks

The goal is not necessarily to deploy these models unchanged. It is to understand the ideas they contributed.

A Useful Mental Model

The evolution of CNN architecture can be summarized as a sequence of questions.

LeNet-5

Can local shared filters learn visual features effectively?

AlexNet

Can CNNs scale to large, realistic image-recognition problems?

VGG

Can depth and uniform small filters improve feature learning?

ResNet

How can extremely deep networks be optimized reliably?

Inception

How can a network process multiple spatial scales without excessive computation?

Each architecture provides a different answer to a fundamental design problem.

Key Takeaway

The basic components of a CNN are straightforward, but combining them effectively requires architectural insight.

Studying successful networks reveals reusable principles:

• Reduce spatial dimensions gradually.
• Increase channels as representations become deeper.
• Reuse simple blocks.
• Control computational cost with bottlenecks.
• Use shortcut connections to improve optimization.
• Learn features at multiple spatial scales.
• Reuse proven architectures when they fit the task.

LeNet-5, AlexNet, and VGG explain the historical development of standard convolutional architectures. Residual networks show how shortcut connections enable much greater depth, while Inception networks demonstrate efficient multiscale processing.

These architectures are valuable not merely as historical models, but as collections of design ideas that continue to influence computer vision and deep learning more broadly.