Building a Deep Convolutional Neural Network for Image Classification
A deep convolutional neural network transforms an image through a sequence of learned feature maps before producing a final classification.
A typical architecture follows this general pattern:\[ \text{image} \rightarrow \text{convolutional features} \rightarrow \text{downsampled features} \rightarrow \text{high-level representation} \rightarrow \text{prediction} \]
As the network becomes deeper:
- Spatial height and width usually decrease.
- The number of feature channels usually increases.
- Receptive fields grow.
- Features become more abstract.
This article develops a complete convolutional network for binary image classification.
Classification Objective
Suppose the input is an RGB image:\[ X\in\mathbb{R}^{39\times39\times3} \]
The task is binary classification:\[ y= \begin{cases} 1,&\text{if the image contains a cat}\\ 0,&\text{otherwise} \end{cases} \]
The network should produce:\[ \hat{y} = P(y=1\mid X) \]
A sigmoid output unit can be used:\[ \hat{y} = \sigma(z) = \frac{1}{1+e^{-z}} \]
Network Architecture
The example network contains:
- A \(3\times3\) convolution with 10 filters
- A \(5\times5\) convolution with 20 filters
- A \(5\times5\) convolution with 40 filters
- A flattening operation
- A binary classification unit
The complete sequence of shapes is:\[ 39\times39\times3 \rightarrow 37\times37\times10 \rightarrow 17\times17\times20 \rightarrow 7\times7\times40 \rightarrow 1960 \rightarrow 1 \]
Input Layer
The input activation is:\[ A^{(0)}=X \]
with dimensions:\[ A^{(0)} \in \mathbb{R}^{39\times39\times3} \]
Therefore:\[ n_H^{(0)}=39 \]\[ n_W^{(0)}=39 \]\[ n_C^{(0)}=3 \]
The three channels correspond to red, green, and blue.
First Convolutional Layer
The first layer uses:
- Filter size: \(3\times3\)
- Stride: 1
- Padding: 0
- Number of filters: 10
Thus:\[ f^{(1)}=3 \]\[ s^{(1)}=1 \]\[ p^{(1)}=0 \]\[ n_C^{(1)}=10 \]
First-Layer Spatial Dimensions
The convolutional output-size formula is:\[ n_H^{(l)} = \left\lfloor \frac{ n_H^{(l-1)} + 2p^{(l)} – f^{(l)} }{ s^{(l)} } \right\rfloor +1 \]
For the first layer:\[ n_H^{(1)} = \left\lfloor \frac{ 39+2(0)-3 }{ 1 } \right\rfloor +1 \]\[ = 36+1 = 37 \]
The width is calculated identically:\[ n_W^{(1)}=37 \]
Because the layer contains 10 filters:\[ n_C^{(1)}=10 \]
Therefore:\[ A^{(1)} \in \mathbb{R}^{37\times37\times10} \]
The transformation is:\[ 39\times39\times3 \longrightarrow 37\times37\times10 \]
Valid, Not Same, Convolution
This layer uses no padding:\[ p^{(1)}=0 \]
It is therefore a valid convolution, not a same convolution.
A same convolution with a \(3\times3\) filter and stride 1 would normally use:\[ p=1 \]
and would preserve the \(39\times39\) spatial dimensions.
Because this example produces \(37\times37\), the correct description is valid convolution.
First-Layer Filter Dimensions
Each filter must span all three input channels:\[ W_k^{(1)} \in \mathbb{R}^{3\times3\times3} \]
There are 10 filters, so the complete weight tensor has shape:\[ W^{(1)} \in \mathbb{R}^{3\times3\times3\times10} \]
The bias contains one value per output channel:\[ b^{(1)} \in \mathbb{R}^{10} \]
First-Layer Parameter Count
Each filter contains:\[ 3\cdot3\cdot3=27 \]
weights and one bias.
With 10 filters:\[ \text{parameters} = 10(27+1) \]\[ =280 \]
First-Layer Computation
The layer computes:\[ Z^{(1)} = \operatorname{Conv} \left( A^{(0)},W^{(1)} \right) + b^{(1)} \]
and then applies an activation function:\[ A^{(1)} = g^{(1)} \left( Z^{(1)} \right) \]
Using ReLU:\[ A^{(1)} = \max\left(0,Z^{(1)}\right) \]
The 10 channels can learn different low-level features, such as:
- Vertical edges
- Horizontal edges
- Diagonal edges
- Color transitions
- Corners
- Simple textures
Second Convolutional Layer
The second layer receives:\[ A^{(1)} \in \mathbb{R}^{37\times37\times10} \]
It uses:
- Filter size: \(5\times5\)
- Stride: 2
- Padding: 0
- Number of filters: 20
Therefore:\[ f^{(2)}=5 \]\[ s^{(2)}=2 \]\[ p^{(2)}=0 \]\[ n_C^{(2)}=20 \]
Second-Layer Spatial Dimensions
The output height is:\[ n_H^{(2)} = \left\lfloor \frac{ 37+2(0)-5 }{ 2 } \right\rfloor +1 \]\[ = \left\lfloor \frac{32}{2} \right\rfloor +1 \]\[ =16+1=17 \]
Similarly:\[ n_W^{(2)}=17 \]
Because 20 filters are used:\[ n_C^{(2)}=20 \]
Thus:\[ A^{(2)} \in \mathbb{R}^{17\times17\times20} \]
The transformation is:\[ 37\times37\times10 \longrightarrow 17\times17\times20 \]
The stride of 2 causes the spatial dimensions to decrease much faster than a stride-1 convolution would.
Second-Layer Filter Dimensions
Each filter spans all 10 channels of the previous layer:\[ W_k^{(2)} \in \mathbb{R}^{5\times5\times10} \]
Because there are 20 filters:\[ W^{(2)} \in \mathbb{R}^{5\times5\times10\times20} \]
The bias has shape:\[ b^{(2)} \in \mathbb{R}^{20} \]
Second-Layer Parameter Count
Each filter has:\[ 5\cdot5\cdot10=250 \]
weights and one bias.
With 20 filters:\[ \text{parameters} = 20(250+1) \]\[ = 20\cdot251 = 5{,}020 \]
Third Convolutional Layer
The third layer receives:\[ A^{(2)} \in \mathbb{R}^{17\times17\times20} \]
It uses:
- Filter size: \(5\times5\)
- Stride: 2
- Padding: 0
- Number of filters: 40
Therefore:\[ f^{(3)}=5 \]\[ s^{(3)}=2 \]\[ p^{(3)}=0 \]\[ n_C^{(3)}=40 \]
Third-Layer Spatial Dimensions
The output height is:\[ n_H^{(3)} = \left\lfloor \frac{ 17+2(0)-5 }{ 2 } \right\rfloor +1 \]\[ = \left\lfloor \frac{12}{2} \right\rfloor +1 \]\[ =6+1=7 \]
Similarly:\[ n_W^{(3)}=7 \]
The output has 40 channels:\[ n_C^{(3)}=40 \]
Therefore:\[ A^{(3)} \in \mathbb{R}^{7\times7\times40} \]
The transformation is:\[ 17\times17\times20 \longrightarrow 7\times7\times40 \]
Third-Layer Parameter Count
Each filter contains:\[ 5\cdot5\cdot20=500 \]
weights and one bias.
With 40 filters:\[ \text{parameters} = 40(500+1) \]\[ = 40\cdot501 = 20{,}040 \]
Feature Progression Across the Network
The complete convolutional transformation is:\[ 39\times39\times3 \rightarrow 37\times37\times10 \rightarrow 17\times17\times20 \rightarrow 7\times7\times40 \]
The spatial dimensions decrease:\[ 39 \rightarrow 37 \rightarrow 17 \rightarrow 7 \]
The channel count increases:\[ 3 \rightarrow 10 \rightarrow 20 \rightarrow 40 \]
This is a common pattern in convolutional architectures.
Why Spatial Dimensions Decrease
Reducing height and width:
- Lowers computational cost
- Reduces activation memory
- Increases effective receptive fields
- Allows deeper features to summarize larger image regions
- Gradually removes unnecessary positional precision
In this example, downsampling is performed with strided convolutions rather than pooling.
Why Channel Counts Increase
As the representation becomes spatially smaller, the network can represent more feature types through additional channels.
Early channels may respond to:
- Edges
- Corners
- Color contrasts
Middle channels may respond to:
- Textures
- Curves
- Simple shapes
Deeper channels may respond to:
- Eyes
- Ears
- Whiskers
- Faces
- Object parts
- Higher-level visual configurations
The increasing channel dimension supports a richer collection of learned features.
Flattening the Final Volume
The final convolutional output is:\[ A^{(3)} \in \mathbb{R}^{7\times7\times40} \]
The number of activations is:\[ 7\cdot7\cdot40 = 1{,}960 \]
Flattening rearranges these values into a vector:\[ a_{\text{flat}} \in \mathbb{R}^{1960} \]
The operation changes the shape but not the values:\[ 7\times7\times40 \longrightarrow 1960 \]
Flattening itself has no trainable parameters.
Binary Classification Output
For binary classification, the flattened representation can be connected to one output unit:\[ z^{(4)} = \left(w^{(4)}\right)^T a_{\text{flat}} + b^{(4)} \]
where:\[ w^{(4)} \in \mathbb{R}^{1960} \]
and:\[ b^{(4)} \in \mathbb{R} \]
The final probability is:\[ \hat{y} = \sigma\left(z^{(4)}\right) \]
where:\[ \sigma(z) = \frac{1}{1+e^{-z}} \]
The prediction can be obtained using a threshold:\[ \hat{c} = \begin{cases} 1,&\hat{y}\ge 0.5\\ 0,&\hat{y}<0.5 \end{cases} \]
Output-Layer Parameter Count
The binary output layer contains:\[ 1960 \]
weights and one bias:\[ 1960+1=1961 \]
parameters.
Multiclass Classification Output
If the network must classify an image into one of \(K\) mutually exclusive categories, the final layer can use \(K\) output units:\[ z = W_{\text{out}}a_{\text{flat}} + b_{\text{out}} \]
where:\[ W_{\text{out}} \in \mathbb{R}^{K\times1960} \]\[ b_{\text{out}} \in \mathbb{R}^{K} \]
The softmax probabilities are:\[ \hat{y}_k = \frac{e^{z_k}} { \sum_{j=1}^{K}e^{z_j} } \]
The predicted class is:\[ \hat{k} = \arg\max_k\hat{y}_k \]
Total Parameter Count
For the binary classifier, the parameter counts are:
| Layer | Parameters |
|---|---|
| First convolution | 280 |
| Second convolution | 5,020 |
| Third convolution | 20,040 |
| Binary output | 1,961 |
| Total | 27,301 |
Thus:\[ 280+5{,}020+20{,}040+1{,}961 = 27{,}301 \]
The network processes an image containing:\[ 39\cdot39\cdot3 = 4{,}563 \]
input values using a compact set of spatially shared filters.
Architecture Summary
| Stage | Filter | Stride | Padding | Filters | Output shape |
|---|---|---|---|---|---|
| Input | — | — | — | — | \(39\times39\times3\) |
| Conv 1 | \(3\times3\) | 1 | 0 | 10 | \(37\times37\times10\) |
| Conv 2 | \(5\times5\) | 2 | 0 | 20 | \(17\times17\times20\) |
| Conv 3 | \(5\times5\) | 2 | 0 | 40 | \(7\times7\times40\) |
| Flatten | — | — | — | — | \(1960\) |
| Sigmoid | — | — | — | 1 | \(1\) |
Forward Propagation
The network’s forward pass can be summarized as:\[ Z^{(1)} = \operatorname{Conv} \left( A^{(0)},W^{(1)} \right) + b^{(1)} \]\[ A^{(1)} = g\left(Z^{(1)}\right) \]\[ Z^{(2)} = \operatorname{Conv} \left( A^{(1)},W^{(2)} \right) + b^{(2)} \]\[ A^{(2)} = g\left(Z^{(2)}\right) \]\[ Z^{(3)} = \operatorname{Conv} \left( A^{(2)},W^{(3)} \right) + b^{(3)} \]\[ A^{(3)} = g\left(Z^{(3)}\right) \]\[ a_{\text{flat}} = \operatorname{Flatten} \left( A^{(3)} \right) \]\[ \hat{y} = \sigma \left( w_{\text{out}}^T a_{\text{flat}} + b_{\text{out}} \right) \]
Training the Network
For binary classification, the loss for one example is:\[ \mathcal{L}(y,\hat{y}) = – \left[ y\log\hat{y} + (1-y)\log(1-\hat{y}) \right] \]
The average training cost is:\[ J = \frac{1}{m} \sum_{i=1}^{m} \mathcal{L} \left( y^{(i)},\hat{y}^{(i)} \right) \]
Backpropagation computes gradients for:
- Every convolutional filter
- Every convolutional bias
- The output weights
- The output bias
An optimization algorithm then updates these parameters to reduce the cost.
Three Common CNN Layer Types
A traditional convolutional neural network commonly combines three layer categories.
Convolutional Layers
A convolutional layer:
- Learns spatial filters
- Shares parameters across positions
- Can change the number of channels
- May preserve or reduce spatial dimensions
A common abbreviation is:\[ \text{CONV} \]
Pooling Layers
A pooling layer:
- Summarizes local regions
- Usually reduces height and width
- Preserves the channel count
- Has no trainable filter weights
A common abbreviation is:\[ \text{POOL} \]
Fully Connected Layers
A fully connected layer receives a vector and computes:\[ z=Wa+b \]
It is commonly used near the prediction end of traditional CNN architectures.
A common abbreviation is:\[ \text{FC} \]
A conventional architecture might therefore look like:\[ \text{INPUT} \rightarrow \text{CONV} \rightarrow \text{POOL} \rightarrow \text{CONV} \rightarrow \text{POOL} \rightarrow \text{FC} \rightarrow \text{OUTPUT} \]
Modern Alternatives to Flattening
Flattening is straightforward, but it can produce a large dense layer when the final feature map is large.
A common modern alternative is global average pooling:\[ 7\times7\times40 \longrightarrow 1\times1\times40 \]
For each channel \(c\):\[ v_c = \frac{1}{49} \sum_{i=1}^{7} \sum_{j=1}^{7} A^{(3)}_{i,j,c} \]
This creates a 40-dimensional vector rather than a 1,960-dimensional vector.
Potential advantages include:
- Fewer output-layer parameters
- Reduced overfitting risk
- Lower memory use
- A direct summary of each learned channel
Flattening remains useful, but it is not the only way to connect convolutional features to a classifier.
Design Decisions
Constructing a CNN requires choosing:
- Number of layers
- Filter sizes
- Number of filters
- Strides
- Padding
- Pooling operations
- Activation functions
- Classification head
- Regularization methods
These values determine:
- Output dimensions
- Parameter count
- Computational cost
- Memory use
- Receptive fields
- Model capacity
Shape Validation
Before implementing a network, verify every layer’s dimensions.
For each convolutional layer:\[ n_H^{(l)} = \left\lfloor \frac{ n_H^{(l-1)} + 2p_H^{(l)} – f_H^{(l)} }{ s_H^{(l)} } \right\rfloor +1 \]\[ n_W^{(l)} = \left\lfloor \frac{ n_W^{(l-1)} + 2p_W^{(l)} – f_W^{(l)} }{ s_W^{(l)} } \right\rfloor +1 \]\[ n_C^{(l)} = \text{number of filters in layer }l \]
For this network:\[ 39 \rightarrow 37 \rightarrow 17 \rightarrow 7 \]
All filter positions fit exactly, so no incomplete final window must be discarded.
Common Mistakes
Calling the first layer a same convolution
The first layer uses no padding and shrinks from 39 to 37. It is a valid convolution.
Forgetting that filter depth matches input channels
The second-layer filters have shape:\[ 5\times5\times10 \]
not \(5\times5\times3\).
The third-layer filters have shape:\[ 5\times5\times20 \]
Setting output channels equal to input channels
The number of output channels is determined by the number of filters, not by the number of input channels.
Forgetting the stride in the output formula
The second and third layers use stride 2, which causes substantial downsampling.
Flattening incorrectly
The final volume contains:\[ 7\cdot7\cdot40=1960 \]
values, not \(7+7+40\).
Assuming flattening learns parameters
Flattening only rearranges the data. The following dense output layer contains the trainable parameters.
Key Takeaway
A convolutional neural network gradually transforms an image into a compact, high-level representation.
In this example:\[ 39\times39\times3 \rightarrow 37\times37\times10 \rightarrow 17\times17\times20 \rightarrow 7\times7\times40 \rightarrow 1960 \rightarrow 1 \]
The spatial dimensions decrease:\[ 39\rightarrow37\rightarrow17\rightarrow7 \]
while the number of channels increases:\[ 3\rightarrow10\rightarrow20\rightarrow40 \]
This pattern allows the network to trade fine spatial resolution for an increasingly rich collection of abstract visual features. The final representation can then be flattened or globally pooled and passed to a sigmoid or softmax classifier.