Padding in Convolutional Neural Networks

Applying a convolution without padding reduces the spatial dimensions of an image or feature map. It also causes pixels near the boundary to participate in fewer convolution calculations than pixels near the center.

Padding addresses these problems by adding extra pixels around the input before applying the convolution.

The two most common configurations are:

• Valid convolution: no padding
• Same convolution: enough padding to preserve the spatial dimensions when the stride is 1

Why Convolution Shrinks an Image

Suppose an input image has dimensions:\[ n\times n \]

and a filter has dimensions:\[ f\times f \]

Assume:

• No padding
• A stride of 1

The filter must remain completely inside the image. Therefore, the number of valid horizontal and vertical positions is:\[ n-f+1 \]

The output dimensions are:\[ (n-f+1)\times(n-f+1) \]

For example, a \(6\times6\) image convolved with a \(3\times3\) filter produces:\[ 6-3+1=4 \]

and therefore:\[ 6\times6 * 3\times3 \longrightarrow 4\times4 \]

This loss of spatial size occurs every time an unpadded convolution is applied.

Problem 1: Repeated Spatial Shrinkage

Suppose several \(3\times3\) convolutions are applied consecutively without padding:\[ 32\times32 \rightarrow 30\times30 \rightarrow 28\times28 \rightarrow 26\times26 \rightarrow\cdots \]

Each layer removes one row or column from every side of the feature map.

A deep network would quickly lose its spatial resolution. Eventually, the feature map could become too small to support additional convolutional layers.

This is undesirable because deep networks often need to process features through many layers before intentionally reducing their resolution.

Problem 2: Boundary Information Is Underused

Pixels near the center participate in many convolution windows. Pixels near the boundary participate in fewer.

Consider a \(3\times3\) filter.

• A corner pixel participates in only one valid filter position.
• An edge pixel participates in a small number of positions.
• A central pixel can participate in as many as nine positions.

For an interior pixel, the filter can be centered around it in several ways. For a corner pixel, most of those positions would place part of the filter outside the image.

As a result, an unpadded convolution gives boundary information less influence over the output.

This can be harmful because important objects may appear near an image boundary.

Adding Padding

Padding adds a border around the input.

If the padding size is:\[ p \]

then \(p\) rows are added above and below the image, and \(p\) columns are added to the left and right.

An \(n\times n\) input becomes:\[ (n+2p)\times(n+2p) \]

For example, padding a \(6\times6\) image with \(p=1\) produces:\[ 6+2(1)=8 \]

so the padded input has dimensions:\[ 8\times8 \]

A \(3\times3\) convolution then produces:\[ 8-3+1=6 \]

Therefore:\[ 6\times6 \overset{p=1}{\longrightarrow} 8\times8 \overset{3\times3\text{ filter}}{\longrightarrow} 6\times6 \]

The original spatial dimensions are preserved.

Zero Padding

The standard approach is to surround the input with zeros.

Suppose the original image is:\[ X= \begin{bmatrix} x_{11}&x_{12}&x_{13}\\ x_{21}&x_{22}&x_{23}\\ x_{31}&x_{32}&x_{33} \end{bmatrix} \]

With \(p=1\), zero padding produces:\[ X_{\text{pad}} = \begin{bmatrix} 0&0&0&0&0\\ 0&x_{11}&x_{12}&x_{13}&0\\ 0&x_{21}&x_{22}&x_{23}&0\\ 0&x_{31}&x_{32}&x_{33}&0\\ 0&0&0&0&0 \end{bmatrix} \]

The padded values are not learned parameters. They are fixed values inserted before the convolution is computed.

Output Dimensions with Padding

For a square input of size \(n\times n\), a square filter of size \(f\times f\), padding \(p\), and stride 1, the output size is:\[ n_{\text{out}} = n+2p-f+1 \]

Therefore, the output dimensions are:\[ (n+2p-f+1) \times (n+2p-f+1) \]

For example:\[ n=6,\qquad f=3,\qquad p=1 \]

Then:\[ n_{\text{out}} = 6+2(1)-3+1 = 6 \]

The output has the same spatial dimensions as the input.

Rectangular Inputs and Filters

For an input with height \(n_H\) and width \(n_W\), filter dimensions \(f_H\times f_W\), and padding \(p_H,p_W\), the output dimensions are:\[ n_H^{\text{out}} = n_H+2p_H-f_H+1 \]\[ n_W^{\text{out}} = n_W+2p_W-f_W+1 \]

These formulas assume:

• Symmetric padding
• Stride 1
• No dilation

The output tensor has spatial shape:\[ n_H^{\text{out}} \times n_W^{\text{out}} \]

Valid Convolution

A valid convolution uses no padding:\[ p=0 \]

The output size is therefore:\[ n_{\text{out}} = n-f+1 \]

For a \(6\times6\) input and a \(3\times3\) filter:\[ n_{\text{out}} = 6-3+1 = 4 \]

Therefore:\[ 6\times6 \longrightarrow 4\times4 \]

The term “valid” means that the filter is applied only where it fits completely inside the original input.

It does not mean that valid convolution is always the correct architectural choice.

Same Convolution

A same convolution aims to preserve the input’s spatial dimensions.

For stride 1, we require:\[ n_{\text{out}}=n \]

Starting with:\[ n_{\text{out}} = n+2p-f+1 \]

set the output size equal to the input size:\[ n+2p-f+1=n \]

Cancel \(n\) from both sides:\[ 2p-f+1=0 \]

Therefore:\[ 2p=f-1 \]

and:\[ p=\frac{f-1}{2} \]

This produces symmetric integer padding when \(f\) is odd.

Same-Padding Examples

A \(3\times3\) filter

\[ f=3 \]

Therefore:\[ p=\frac{3-1}{2}=1 \]

Pad the input with one pixel on every side.

A \(5\times5\) filter

\[ f=5 \]

Therefore:\[ p=\frac{5-1}{2}=2 \]

Pad with two pixels on every side.

A \(7\times7\) filter

\[ f=7 \]

Therefore:\[ p=\frac{7-1}{2}=3 \]

Pad with three pixels on every side.

These values assume a stride of 1.

Why Odd-Sized Filters Are Common

Convolutional networks frequently use odd filter sizes such as:

• \(1\times1\)
• \(3\times3\)
• \(5\times5\)
• \(7\times7\)

There are two practical reasons.

Odd filters have a central position

A \(3\times3\) filter has a unique center:\[ \begin{bmatrix} \cdot&\cdot&\cdot\\ \cdot&\boxed{\cdot}&\cdot\\ \cdot&\cdot&\cdot \end{bmatrix} \]

This makes it natural to describe the filter as operating around a particular input location.

A \(5\times5\) or \(7\times7\) filter also has a unique central element.

An even-sized filter, such as \(2\times2\), has no single central cell.

Odd filters support symmetric same padding

For an odd filter:\[ p=\frac{f-1}{2} \]

is an integer.

For example:\[ f=3\Rightarrow p=1 \]

For an even filter such as \(f=4\):\[ p=\frac{4-1}{2}=1.5 \]

Symmetric integer padding cannot satisfy this. Preserving the output size requires asymmetric padding, such as one pixel on one side and two on the other.

Even-sized filters are valid and sometimes useful, but odd-sized filters make alignment and symmetric padding more straightforward.

Same Padding with Even-Sized Filters

Modern deep learning libraries can implement same padding for even filters by distributing an unequal number of padding values across the two sides.

For a one-dimensional example with \(f=4\) and stride 1, the total required padding is:\[ f-1=3 \]

A library might use:\[ p_{\text{left}}=1, \qquad p_{\text{right}}=2 \]

or the reverse, depending on its convention.

Therefore, “same” does not always imply identical padding on both sides. It means that the output-size rule is preserved.

How Padding Improves Boundary Usage

Without padding, a corner pixel participates in only one \(3\times3\) convolution window.

With \(p=1\), the filter can also be positioned around that corner using zeros outside the original image. The corner pixel participates in more output calculations.

Padding does not make boundary pixels participate in exactly the same visual context as central pixels—the surrounding values are partly artificial—but it reduces the severe underrepresentation created by valid convolution.

Padding Does Not Add Real Information

Zero padding preserves dimensions, but the added zeros do not contain image information.

This has several implications:

• Boundary calculations use less real context.
• Repeated padding can introduce boundary artifacts.
• Features near the edges may behave differently from features in the interior.
• A large padding region can dominate the receptive field near a boundary.

Padding reduces information loss; it does not create missing visual content.

Other Padding Modes

Although zero padding is standard, other padding strategies exist.

Reflection Padding

Reflection padding mirrors values near the boundary.

For a one-dimensional signal:

$[a, b, c]$

a reflected extension might resemble:

$[c, b, a, b, c, b, a]$

This can reduce abrupt artificial transitions at the border.

Replication Padding

Replication padding repeats boundary values:

$[a, b, c] \rightarrow [a, a, a, b, c, c, c]$

This avoids introducing zeros but can create flat extended regions.

Circular Padding

Circular padding wraps values from one side to the other:

$[a, b, c] \rightarrow [b, c, a, b, c, a, b]$

This is useful when the data is naturally periodic.

Choosing a Padding Mode

The correct choice depends on the data and task. For ordinary image classification, zero padding is the common default.

Padding a Multichannel Input

For an RGB image:\[ X \in \mathbb{R}^{n_H\times n_W\times3} \]

padding is applied to the height and width dimensions, not normally to the channel dimension.

With padding \(p\):\[ X_{\text{pad}} \in \mathbb{R}^{(n_H+2p)\times(n_W+2p)\times3} \]

The number of channels remains unchanged.

For a batch:\[ X \in \mathbb{R}^{m\times n_H\times n_W\times n_C} \]

the padded batch has shape:\[ X_{\text{pad}} \in \mathbb{R}^{ m\times(n_H+2p)\times(n_W+2p)\times n_C } \]

assuming a channels-last layout.

A NumPy Padding Example

import numpy as np

def zero_pad(images, pad):
    """
    images shape:
        (batch_size, height, width, channels)
    """
    return np.pad(
        images,
        pad_width=(
            (0, 0),
            (pad, pad),
            (pad, pad),
            (0, 0),
        ),
        mode="constant",
        constant_values=0,
    )

import numpy as np

def zero_pad(images, pad):
    """
    images shape:
        (batch_size, height, width, channels)
    """
    return np.pad(
        images,
        pad_width=(
            (0, 0),
            (pad, pad),
            (pad, pad),
            (0, 0),
        ),
        mode="constant",
        constant_values=0,
    )

Example:

images = np.random.randn(4, 6, 6, 3)
padded_images = zero_pad(images, pad=1)

print(images.shape)
print(padded_images.shape)

images = np.random.randn(4, 6, 6, 3)
padded_images = zero_pad(images, pad=1)

print(images.shape)
print(padded_images.shape)

Output:

(4, 6, 6, 3)
(4, 8, 8, 3)

(4, 6, 6, 3)
(4, 8, 8, 3)

The batch size and channel count remain unchanged. Only height and width increase.

Valid and Same Padding in Practice

Conceptually, a convolutional layer can be configured as:

# Spatial dimensions shrink when the filter is larger than 1.
Conv2D(filters=64, kernel_size=3, padding="valid")

# Spatial dimensions are preserved when stride is 1.
Conv2D(filters=64, kernel_size=3, padding="same")

# Spatial dimensions shrink when the filter is larger than 1.
Conv2D(filters=64, kernel_size=3, padding="valid")

# Spatial dimensions are preserved when stride is 1.
Conv2D(filters=64, kernel_size=3, padding="same")

The exact tensor layout and interface depend on the library, but the mathematical distinction remains the same.

Why Same Padding Is Useful in Deep Networks

Same padding allows a network to apply many convolutional transformations without automatically shrinking the feature maps.

For example:\[ 64\times64 \rightarrow 64\times64 \rightarrow 64\times64 \rightarrow 64\times64 \]

The network can increase representational depth while preserving spatial resolution.

Spatial reduction can then be introduced deliberately through:

• Strided convolution
• Pooling
• Resizing operations
• Architecture-specific downsampling blocks

This is preferable to losing spatial resolution as an accidental consequence of every convolution.

When Valid Padding Is Useful

Valid convolution remains useful when:

• Shrinking the feature map is intentional.
• Boundary values should not depend on artificial padding.
• The architecture requires exact receptive-field behavior.
• The input already contains an external padded region.
• A later operation expects the reduced dimensions.

The choice between valid and same padding is an architectural decision rather than a universal rule.

Padding and Receptive Fields

Padding preserves the number of spatial positions, but deeper units still obtain information from increasingly large portions of the original image.

For two consecutive \(3\times3\), stride-1 convolutions:

• A unit in the first layer sees a \(3\times3\) region.
• A unit in the second layer sees an effective \(5\times5\) region of the original input.

After another layer, the effective receptive field becomes \(7\times7\).

Thus, same padding preserves feature-map dimensions while allowing receptive fields to grow with depth.

Common Mistakes

Forgetting that padding is added on both sides

Padding \(p\) increases a dimension by:\[ 2p \]

not \(p\).

Using the same-padding formula when stride is not 1

The formula:\[ p=\frac{f-1}{2} \]

assumes stride 1 and an odd filter size.

Padding the channel dimension

Standard image convolution pads height and width, not channels.

Assuming same padding is always symmetric

Even filters or strides greater than one may require asymmetric padding.

Believing padding completely removes boundary effects

Padding improves boundary coverage but does not provide genuine image information beyond the original border.

Key Takeaway

Without padding, convolving an \(n\times n\) input with an \(f\times f\) filter using stride 1 produces:\[ (n-f+1)\times(n-f+1) \]

This causes two problems:

• The feature map shrinks after every convolution.
• Boundary pixels influence fewer outputs than central pixels.

Adding \(p\) pixels of padding gives:\[ n_{\text{out}} = n+2p-f+1 \]

A valid convolution uses:\[ p=0 \]

A same convolution with stride 1 and an odd-sized filter uses:\[ p=\frac{f-1}{2} \]

Padding allows deep convolutional networks to preserve spatial dimensions, retain more boundary information, and control downsampling deliberately rather than shrinking after every layer.