Convolution and Edge Detection in Images
The convolution operation is one of the fundamental building blocks of a convolutional neural network. It allows a model to detect local visual patterns such as edges, corners, textures, and eventually more complicated object structures.
Edge detection provides a simple way to understand how convolution works.
Why Detect Edges?
Objects are often distinguished by boundaries where pixel intensities change sharply.
An image may contain:
- Vertical edges along the sides of buildings
- Horizontal edges along railings or the horizon
- Curved edges around faces
- Diagonal edges along roads or rooftops
Early convolutional layers commonly learn filters that respond to these simple visual patterns. Deeper layers combine them into increasingly complex representations:\[ \text{pixels} \rightarrow \text{edges} \rightarrow \text{textures} \rightarrow \text{object parts} \rightarrow \text{complete objects} \]
For example, a face-recognition network might gradually learn:\[ \text{edges} \rightarrow \text{eyes, nose, and mouth} \rightarrow \text{face} \rightarrow \text{identity representation} \]
Representing a Grayscale Image
Consider a grayscale image with height and width equal to 6:\[ X\in\mathbb{R}^{6\times6} \]
Because it is grayscale, each pixel has one intensity value. It could also be written with an explicit channel dimension:\[ X\in\mathbb{R}^{6\times6\times1} \]
An RGB image would instead have three channels:\[ X_{\text{RGB}}\in\mathbb{R}^{6\times6\times3} \]
To keep the first example simple, we will use a two-dimensional grayscale matrix.
What Is a Filter?
A filter, also called a kernel, is a small matrix that scans across an image.
A vertical-edge filter can be written as:\[ K= \begin{bmatrix} 1 & 0 & -1\\ 1 & 0 & -1\\ 1 & 0 & -1 \end{bmatrix} \]
This filter looks for image regions that have:
- Larger intensity values on the left
- Smaller intensity values on the right
- A strong transition between them
It can therefore respond strongly to a vertical boundary.
The filter has shape:\[ K\in\mathbb{R}^{3\times3} \]
Correcting a Common Transcription Error
A filter written as\[ \begin{bmatrix} 1 & 1 & 1\\ 0 & 0 & 0\\ -1 & -1 & -1 \end{bmatrix} \]
responds primarily to horizontal edges, because it compares pixels above and below one another.
By contrast,\[ \begin{bmatrix} 1 & 0 & -1\\ 1 & 0 & -1\\ 1 & 0 & -1 \end{bmatrix} \]
responds to vertical edges, because it compares pixels on the left and right.
| Filter structure | Primary response |
|---|---|
| Positive left, negative right | Vertical edges |
| Positive top, negative bottom | Horizontal edges |
Sliding the Filter Across the Image
The filter is placed over a local \(3\times3\) region of the image. The corresponding values are multiplied element by element and then added.
Suppose the upper-left image region is:\[ X_{1:3,1:3} = \begin{bmatrix} 3 & 0 & 1\\ 1 & 5 & 8\\ 2 & 7 & 2 \end{bmatrix} \]
Using the vertical-edge filter:\[ K= \begin{bmatrix} 1 & 0 & -1\\ 1 & 0 & -1\\ 1 & 0 & -1 \end{bmatrix} \]
the first output value is:\[ \begin{aligned} Z_{1,1} ={}& 3(1)+0(0)+1(-1)\\ &+1(1)+5(0)+8(-1)\\ &+2(1)+7(0)+2(-1) \end{aligned} \]
Therefore:\[ Z_{1,1} = 3-1+1-8+2-2 = -5 \]
This value is placed in the upper-left position of the output matrix.
Elementwise Product and Summation
The local calculation can be expressed using the Frobenius inner product:\[ Z_{i,j} = \left\langle X_{i:i+f-1,\;j:j+f-1}, K \right\rangle_F \]
Equivalently:\[ Z_{i,j} = \sum_{a=0}^{f-1} \sum_{b=0}^{f-1} X_{i+a,j+b}K_{a,b} \]
where \(f\) is the filter size.
For a \(3\times3\) filter:\[ f=3 \]
The calculation is repeated after moving the filter across the image.
Moving One Position at a Time
After computing the first output value, move the filter one column to the right:\[ X_{1:3,2:4} \]
Compute the same elementwise multiplication and summation to obtain:\[ Z_{1,2} \]
Continue moving right until the filter reaches the edge of the image.
Then move it down by one row and repeat:\[ X_{2:4,1:3}, \quad X_{2:4,2:4}, \quad \ldots \]
This scanning process produces the complete output feature map.
Output Dimensions
Suppose the input is:\[ n\times n \]
and the filter is:\[ f\times f \]
If there is no padding and the filter moves one position at a time, the output dimensions are:\[ (n-f+1)\times(n-f+1) \]
For a \(6\times6\) image and a \(3\times3\) filter:\[ 6-3+1=4 \]
Therefore:\[ 6\times6 * 3\times3 \longrightarrow 4\times4 \]
The output is a \(4\times4\) feature map.
General Output-Size Formula
If the input size is \(n_H\times n_W\), the filter size is \(f_H\times f_W\), and there is no padding with stride 1, then:\[ n_H^{\text{out}} = n_H-f_H+1 \]\[ n_W^{\text{out}} = n_W-f_W+1 \]
Therefore:\[ Z \in \mathbb{R}^{ (n_H-f_H+1) \times (n_W-f_W+1) } \]
Padding and stride modify this formula, but the basic edge-detection example uses no padding and a stride of one.
A Simplified Vertical-Edge Example
Consider an image whose left half is bright and whose right half is dark:\[ X= \begin{bmatrix} 10&10&10&0&0&0\\ 10&10&10&0&0&0\\ 10&10&10&0&0&0\\ 10&10&10&0&0&0\\ 10&10&10&0&0&0\\ 10&10&10&0&0&0 \end{bmatrix} \]
Visually, the image contains a strong vertical boundary:
Bright region | Dark region ↑ vertical edge
Apply the vertical-edge filter:\[ K= \begin{bmatrix} 1&0&-1\\ 1&0&-1\\ 1&0&-1 \end{bmatrix} \]
Response in a Uniform Region
When the filter covers only bright pixels, the calculation is:\[ (10+10+10)-(10+10+10)=0 \]
The filter produces zero because the left and right sides of the local region have the same intensity.
Similarly, when it covers only dark pixels:\[ (0+0+0)-(0+0+0)=0 \]
A nearly uniform region does not look like an edge.
Response at the Boundary
When the filter crosses the boundary, its positive column covers bright pixels while its negative column covers dark pixels:\[ (10+10+10)-(0+0+0)=30 \]
The large positive response indicates a strong vertical edge.
The complete result is:\[ Z= \begin{bmatrix} 0&30&30&0\\ 0&30&30&0\\ 0&30&30&0\\ 0&30&30&0 \end{bmatrix} \]
The two-column-wide response occurs because the example uses a very small image and a \(3\times3\) filter. On a high-resolution image, the detected boundary appears much narrower relative to the image dimensions.
Interpreting the Output Feature Map
The output matrix is called a feature map or activation map.
Large positive values indicate a strong match with the filter’s pattern:\[ \text{bright on left} \rightarrow \text{dark on right} \]
Values near zero indicate little evidence for that pattern.
Negative values indicate the opposite transition:\[ \text{dark on left} \rightarrow \text{bright on right} \]
Thus, the sign carries directional information.
| Output value | Interpretation |
|---|---|
| Large positive | Strong edge in the filter’s preferred direction |
| Near zero | No strong matching edge |
| Large negative | Strong edge in the opposite direction |
Reversing the Edge Direction
Consider the reversed filter:\[ K_{\text{reverse}} = \begin{bmatrix} -1&0&1\\ -1&0&1\\ -1&0&1 \end{bmatrix} \]
This filter responds positively when the image changes from dark on the left to bright on the right.
The two filters detect the same orientation but opposite intensity transitions.
If direction does not matter, the system may examine the magnitude:\[ |Z_{i,j}| \]
or combine responses from several learned filters.
Horizontal Edge Detection
A corresponding horizontal-edge filter is:\[ K_H= \begin{bmatrix} 1&1&1\\ 0&0&0\\ -1&-1&-1 \end{bmatrix} \]
It compares the upper and lower portions of each local image region.
A strong positive value indicates:\[ \text{bright above} \rightarrow \text{dark below} \]
A strong negative value indicates the reverse transition.
Vertical and Horizontal Filters
The two filters are:\[ K_V= \begin{bmatrix} 1&0&-1\\ 1&0&-1\\ 1&0&-1 \end{bmatrix} \]\[ K_H= \begin{bmatrix} 1&1&1\\ 0&0&0\\ -1&-1&-1 \end{bmatrix} \]
Their outputs can be interpreted as directional edge responses:\[ G_x=X*K_V \]\[ G_y=X*K_H \]
An approximate edge magnitude can then be constructed as:\[ G= \sqrt{G_x^2+G_y^2} \]
and the edge orientation can be estimated by:\[ \theta = \operatorname{atan2}(G_y,G_x) \]
This connects basic convolution to classical image-gradient calculations.
Convolution vs. Cross-Correlation
There is an important technical distinction between mathematical convolution and the operation commonly used in neural networks.
Mathematical convolution
In strict mathematical convolution, the kernel is flipped before sliding:\[ (X*K)_{i,j} = \sum_a\sum_b X_{i+a,j+b}K_{-a,-b} \]
Cross-correlation
In cross-correlation, the kernel is used without flipping:\[ (X\star K)_{i,j} = \sum_a\sum_b X_{i+a,j+b}K_{a,b} \]
The edge-detection calculation described above is technically cross-correlation.
However, deep learning libraries and literature commonly call it convolution. Because CNN filters are learned, the distinction usually does not affect model capacity: the network can learn the flipped version if necessary.
In most deep learning discussions, “convolution” refers operationally to sliding an unflipped filter across the input.
A NumPy Implementation
A simple two-dimensional operation can be implemented as follows:
import numpy as np
def conv2d_valid(image, kernel):
image_height, image_width = image.shape
kernel_height, kernel_width = kernel.shape
output_height = image_height - kernel_height + 1
output_width = image_width - kernel_width + 1
output = np.zeros((output_height, output_width))
for i in range(output_height):
for j in range(output_width):
region = image[
i:i + kernel_height,
j:j + kernel_width
]
output[i, j] = np.sum(region * kernel)
return outputUsing the simplified edge image:
image = np.array([
[10, 10, 10, 0, 0, 0],
[10, 10, 10, 0, 0, 0],
[10, 10, 10, 0, 0, 0],
[10, 10, 10, 0, 0, 0],
[10, 10, 10, 0, 0, 0],
[10, 10, 10, 0, 0, 0],
], dtype=np.float32)
vertical_filter = np.array([
[1, 0, -1],
[1, 0, -1],
[1, 0, -1],
], dtype=np.float32)
edges = conv2d_valid(image, vertical_filter)
print(edges)The result is:
[[ 0. 30. 30. 0.]
[ 0. 30. 30. 0.]
[ 0. 30. 30. 0.]
[ 0. 30. 30. 0.]]From Hand-Designed Filters to Learned Filters
In classical image processing, filters are selected manually.
Examples include:
- Vertical-edge filters
- Horizontal-edge filters
- Sobel filters
- Sharpening filters
- Blurring filters
In a convolutional neural network, the filter values are usually trainable parameters.
Instead of fixing:\[ K= \begin{bmatrix} 1&0&-1\\ 1&0&-1\\ 1&0&-1 \end{bmatrix} \]
the network begins with initialized parameters:\[ K= \begin{bmatrix} k_{11}&k_{12}&k_{13}\\ k_{21}&k_{22}&k_{23}\\ k_{31}&k_{32}&k_{33} \end{bmatrix} \]
Backpropagation updates these values:\[ K \leftarrow K-\alpha\frac{\partial J}{\partial K} \]
The network may learn filters that detect:
- Edges at different angles
- Color transitions
- Curves
- Repeated textures
- Small object parts
- Patterns that do not have simple human-defined names
This is the transition from classical filtering to convolutional representation learning.
Multiple Filters
A single filter produces one feature map. A convolutional layer normally learns many filters:\[ K^{(1)},K^{(2)},\ldots,K^{(n_C)} \]
Each filter produces a separate output channel:\[ Z^{(k)}=X*K^{(k)}+b^{(k)} \]
Stacking the results gives:\[ Z \in \mathbb{R}^{ n_H^{\text{out}} \times n_W^{\text{out}} \times n_C } \]
One filter may respond to vertical edges, another to horizontal edges, and others to diagonal boundaries or textures.
The model does not need to be told which visual patterns to learn. Training determines which filters are useful for the final objective.
Why Convolution Is Efficient
A \(3\times3\) grayscale filter contains only:\[ 3\cdot3=9 \]
weights.
The same nine values are reused across every spatial location. This is known as parameter sharing.
Without parameter sharing, a separate detector would need to be learned for every possible image location. Convolution instead assumes that a useful feature in one area may also be useful elsewhere.
For an RGB image, a \(3\times3\) filter spans all three channels:\[ 3\times3\times3 \]
and therefore contains:\[ 27 \]
weights, plus a possible bias.
The parameter count depends on filter dimensions and channel counts—not directly on image height or width.
Convolution in Deep Learning Libraries
Deep learning libraries provide optimized convolution operations. They typically expect batched tensors and explicit channel dimensions.
A common conceptual input shape is:\[ (\text{batch},\text{height},\text{width},\text{channels}) \]
or:\[ (\text{batch},\text{channels},\text{height},\text{width}) \]
depending on the framework and configuration.
A convolutional layer generally specifies:
- Number of filters
- Filter size
- Stride
- Padding
- Whether to use a bias
- Activation function
The library performs the sliding operations efficiently using highly optimized CPU or accelerator kernels.
Key Takeaway
Convolution detects local patterns by sliding a small filter across an image. At every position, it:
- Selects a local image region.
- Multiplies it element by element with the filter.
- Adds the products.
- Places the result in an output feature map.
For a \(6\times6\) image convolved with a \(3\times3\) filter using stride 1 and no padding:\[ 6\times6 * 3\times3 \longrightarrow 4\times4 \]
A vertical-edge filter compares pixels on the left and right:\[ \begin{bmatrix} 1&0&-1\\ 1&0&-1\\ 1&0&-1 \end{bmatrix} \]
while a horizontal-edge filter compares pixels above and below:\[ \begin{bmatrix} 1&1&1\\ 0&0&0\\ -1&-1&-1 \end{bmatrix} \]
In a CNN, these filter values are learned through backpropagation. This allows the network to discover the local visual patterns most useful for its task.