Object Detection: Localization, YOLO, IoU, NMS, and R-CNN

An image classifier can label a scene as containing a dog. An object detector must return a separate box, class, and score for each detected dog, including no detections when none are found. Many detectors produce a fixed collection of candidate boxes and then select a variable-length result. This article follows those candidates from coordinate prediction through duplicate removal and evaluation; the examples isolate these operations rather than train a complete detector.

Localization: classification plus regression

For an image containing one object, a localization head can predict a class and four numbers: the box center \((b_x,b_y)\), width \(b_w\), and height \(b_h\). A loss such as \(L=L_{\mathrm{class}}+\lambda L_{\mathrm{box}}\) combines classification and localization, with \(\lambda\) setting their relative weight. Smooth L1 has bounded slope with respect to large coordinate residuals, unlike squared error; IoU-based losses are another option. The choice and target parameterization depend on the detector.

Keep the coordinate convention explicit. In a 200-by-100 image, a box with corners \((20,10,100,50)\) has center \((60,30)\), width 80, and height 40. Dividing horizontal quantities by 200 and vertical quantities by 100 gives normalized center-size coordinates \((0.3,0.3,0.4,0.4)\). Normalization is useful, but pixel coordinates and encoded offsets are also valid. Gradient scale depends on the loss, reduction, and encoding; pixel targets do not automatically dominate classification.

Convolution, padding, stride, and pooling are introduced in CNN Fundamentals: From Convolution to Image Classification.

Landmark detection

A direct landmark-regression head can predict \(2L\) numbers for \(L\) ordered points, such as eyes or joints. Heatmap-based methods instead predict a spatial score map for each point. Missing or unannotated landmarks need a validity mask so they do not contribute an invented coordinate target. Detecting several people also requires associating each set of joints with a person; changing the output dimension alone does not solve that problem.

Use a consistent landmark naming convention. Geometric augmentation must transform coordinates along with pixels, and a flip may require permuting left/right landmark indices according to that convention. For pixel-center coordinates indexed from 0 to \(W-1\), a horizontal flip maps \(x\) to \(W-1-x\). In continuous box-edge coordinates spanning 0 to \(W\), it maps \((x_1,x_2)\) to \((W-x_2,W-x_1)\). A crop may remove landmarks or objects, so update their validity as well. Label errors can affect the loss; the loss curve alone does not reliably identify their cause.

From sliding windows to a single pass

A sliding-window baseline runs a classifier on crops at several positions and scales. This can be expensive because overlapping crops repeat feature computation, and its coverage depends on window sizes and stride. A classifier’s crop label also does not guarantee that the crop tightly localizes the object.

A convolutional feature map lets many candidate locations share computation. A dense layer over a fixed-size feature patch can be rewritten as a convolution with the corresponding kernel; later dense layers become 1-by-1 convolutions. Equivalence to separate crop evaluation requires matching padding, sampling, and other operations. Multiple image scales may still require additional passes. Shared feature extraction is central to the dense and region-based detectors below, whose prediction heads differ.

YOLO’s grid formulation

In original YOLO (v1), the object center selects a cell in an \(S\times S\) grid. Each cell predicts \(B\) boxes with five values each and one shared set of \(C\) conditional class probabilities. Its output is \(S\times S\times(5B+C)\): with \(S=7\), \(B=2\), and \(C=20\), this is \(7\times7\times30\). Box confidence targets combine object presence with localization IoU. The training assignment selects a responsible box predictor for the cell’s object.

Anchor-based YOLO variants instead attach class predictions to each anchor, commonly giving \(S\times S\times B\times(5+C)\) per prediction scale. YOLOv2 uses sigmoid center offsets relative to the grid cell and anchor-scaled widths and heights. For cell column 2, row 1, grid size 4, and offsets \((0.25,0.5)\), the normalized center is \(((2+0.25)/4,(1+0.5)/4)=(0.5625,0.375)\). This encoding is not a specification for every YOLO version.

Original YOLO has one object assignment and shared class predictions per cell, which limits nearby-object detection. Multiple anchor assignments can represent more than one object in a cell, but available slots and matching rules can still conflict. Finer feature maps and predictions at several resolutions help allocate candidates to objects of different sizes. They do not guarantee that every crowded configuration is resolved.

During training, a matching rule assigns ground-truth objects to candidate locations or boxes. In a common anchor-based setup, positive candidates receive class and box targets, negatives receive a background/objectness target, and ignored candidates contribute no assigned loss. Box regression is trained on positives. This handles images with no objects and prevents background locations from being asked to regress arbitrary boxes. Exact losses and normalization differ across architectures.

At inference, decode predicted offsets into boxes, map them back through any resize or padding to the original image, and form class scores according to the model’s definition. For a model using objectness times conditional class probability, 0.8 times 0.75 gives a score of 0.6. A score cutoff filters low-scoring candidates; NMS then handles overlap. The score cutoff, NMS IoU threshold, and evaluation IoU threshold have different roles.

Intersection over Union

For positive-area axis-aligned boxes, \(\operatorname{IoU}(A,B)=|A\cap B|/(|A|+|B|-|A\cap B|)\). Two 10-by-10 boxes overlapping in a 5-by-5 square have IoU \(25/(100+100-25)=1/7\). The code uses continuous corner coordinates \((x_1,y_1,x_2,y_2)\), with no inclusive-pixel +1. It rejects inverted or zero-area boxes. Run the NumPy blocks in order; the exercises reuse these helpers.

import numpy as np

def checked_box(box):
    box = np.asarray(box, dtype=float)
    if (box.shape != (4,) or not np.isfinite(box).all()
            or box[2] <= box[0] or box[3] <= box[1]):
        raise ValueError("expected finite (x1, y1, x2, y2) with positive area")
    return box

def iou(box_a, box_b):
    a, b = checked_box(box_a), checked_box(box_b)
    x1, y1 = max(a[0], b[0]), max(a[1], b[1])
    x2, y2 = min(a[2], b[2]), min(a[3], b[3])
    inter = max(0.0, x2 - x1) * max(0.0, y2 - y1)
    area_a = (a[2] - a[0]) * (a[3] - a[1])
    area_b = (b[2] - b[0]) * (b[3] - b[1])
    return inter / (area_a + area_b - inter)

print(round(iou((0, 0, 10, 10), (5, 5, 15, 15)), 4))
print(iou((0, 0, 10, 10), (0, 0, 10, 10)))
print(iou((0, 0, 10, 10), (20, 20, 30, 30)))
# 0.1429
# 1.0
# 0.0

Each intersection side length must be clamped at zero. If boxes are separated along both axes, two negative lengths multiply to a spurious positive area. Separation along only one axis instead gives a negative area without the clamps. Tests should include both arrangements, touching edges, containment, and identical boxes.

For the loss \(1-\operatorname{IoU}\), strictly separated boxes lie on a flat region: small movements that keep them separated do not change the loss. This is a limitation, not a reason to rule out IoU-based training. GIoU adds a penalty for unused area in the smallest enclosing box; DIoU uses center distance, and CIoU also considers aspect ratio. These terms can guide separated boxes, but none implies a useful nonzero gradient in every configuration. At exact edge contact the function can have a kink, so a single derivative need not exist.

Non-maximum suppression

Several candidates may describe the same object. Greedy hard NMS keeps the highest-scoring box, suppresses remaining boxes with IoU strictly greater than the threshold, and repeats on those left. The function below processes one class and one image. It returns input indices, handles an empty array shaped (0, 4), and breaks score ties by input order. Its simple pairwise implementation is intended for understanding the rule.

def nms(boxes, scores, iou_threshold=0.5):
    boxes, scores = np.asarray(boxes, dtype=float), np.asarray(scores, dtype=float)
    if (boxes.ndim != 2 or boxes.shape[1] != 4
            or scores.shape != (len(boxes),) or not np.isfinite(scores).all()
            or not 0 <= iou_threshold <= 1):
        raise ValueError("expected boxes (N,4), finite scores (N,), threshold in [0,1]")
    for box in boxes:
        checked_box(box)
    order = np.argsort(-scores, kind="stable")
    keep = []
    while order.size > 0:
        best = order[0]
        keep.append(int(best))
        rest = order[1:]
        overlaps = np.array([iou(boxes[best], boxes[j]) for j in rest])
        order = rest[overlaps <= iou_threshold]
    return keep

boxes = np.array([[0, 0, 10, 10], [1, 1, 11, 11], [50, 50, 60, 60]])
scores = np.array([0.9, 0.8, 0.7])
print(nms(boxes, scores, 0.5))
# [0, 2]

Apply this function separately to each class when using class-wise NMS, then map the kept indices back to the original candidates. Class-agnostic NMS is another policy, but can suppress overlapping objects with different labels, such as a person and bicycle. Lower thresholds suppress more overlaps at each comparison; crowded scenes can lose genuine instances. Higher thresholds may leave duplicates. Soft-NMS reduces scores according to overlap instead of immediately deleting boxes; its benefit depends on the data and settings.

Anchor boxes

An anchor supplies a reference center and size for a candidate box. One common encoding uses \(t_x=(x-x_a)/w_a\) and \(t_w=\log(w/w_a)\), with analogous vertical terms. An anchor centered at \(x_a=50\), width \(w_a=20\), and offsets \(t_x=0.1\), \(t_w=\log2\) decodes to center 52 and width 40. This example uses an R-CNN-style encoding; YOLOv2’s center encoding differs. A tall anchor is a geometric prior, not a reserved pedestrian class.

The original Faster R-CNN region proposal network marks anchors positive for IoU above 0.7 or for the highest IoU with a ground-truth box, including ties; non-positive anchors below 0.3 are negative. Other anchors are ignored. These are rules for that proposal stage, not universal detector thresholds. Different architectures use different matching and tie-breaking rules. Multiple anchors can reduce assignment collisions, but cannot guarantee unique slots for every object.

Anchor scales and aspect ratios can be chosen by hand or estimated from training-box dimensions; clustering is one approach. Anchor-free designs remove that shape list. FCOS predicts distances from feature locations to four box edges. The CenterNet described in Objects as Points predicts object centers, sizes, and offsets instead. Such models still need choices about feature resolution, assignment, and training; “anchor-free” does not mean free of design choices.

The R-CNN family

A region proposal is a candidate area that may contain an object. In the R-CNN family, a second stage classifies each proposal and refines its box. Fast R-CNN extracts fixed-size region features from a shared feature map using RoI (region of interest) pooling. Faster R-CNN adds a region proposal network (RPN) on shared features. Mask R-CNN replaces coordinate quantization in RoI pooling with bilinear sampling in RoIAlign and predicts a separate object mask.

One-stage detectors such as SSD and RetinaNet predict classes and boxes densely from feature maps without a separate proposal-classification stage. Two-stage versus one-stage is an architectural distinction, not a fixed accuracy or speed ranking. Compare the actual models at the same input size and evaluation protocol, with latency measured on the intended hardware. RetinaNet’s focal loss reduces the contribution of easy examples, addressing the large number of easy background candidates in dense detection.

How to choose an objective for a given problem is the subject of Loss Function Design: Choosing an Objective That Matches the Problem.

Evaluation with mAP

For a simple evaluation without ignored or crowd annotations, sort detections by score within each class. Match each detection only to ground truth in the same image, using the evaluator’s IoU and one-to-one matching rules. A correctly matched box is a true positive; a duplicate, wrong-class detection, or unmatched detection is a false positive. Missed objects reduce recall. Accumulate precision and recall across the dataset for each class. AP summarizes this ranked curve using the protocol’s interpolation rule, and mAP averages class AP values.

State the full protocol with the result. VOC evaluation uses IoU 0.5, but VOC 2007’s 11-point AP differs from later all-points interpolation. COCO’s main bounding-box AP averages over classes and ten IoU thresholds, 0.50 through 0.95, using 101 recall levels and a default maximum of 100 detections per image. Crowd and ignored annotations have additional matching rules. Report AP50 separately when needed; it is not the same statistic as COCO’s main AP. Use the official evaluator for benchmark comparisons.

For a one-class example with two ground-truth objects, suppose score order produces a correct match, a duplicate of that match, and a correct match to the second object. Precision is 1, 1/2, 2/3 and recall is 1/2, 1/2, 1. With an all-points interpolated precision envelope, AP is (1/2) × 1 + (1/2) × (2/3) = 5/6. The code illustrates this arithmetic; it is not a COCO evaluator.

tp = np.array([1, 0, 1])
fp = 1 - tp
recall = np.cumsum(tp) / 2
precision = np.cumsum(tp) / (np.cumsum(tp) + np.cumsum(fp))
envelope = np.maximum.accumulate(precision[::-1])[::-1]
ap = np.sum(np.diff(np.r_[0.0, recall]) * envelope)
print(np.round(precision, 4))
print(recall)
print(round(ap, 4))
# [1.     0.5    0.6667]
# [0.5 0.5 1. ]
# 0.8333

The encoder-decoder with skip connections is built in Semantic Segmentation and U-Net Explained.

Exercises

1. The clamp that matters. Remove the max(0, ...) clamps from the IoU implementation and compute the IoU for (0,0,10,10) and (20,20,30,30). Report the value and explain why the bug survives casual testing.

You should get: an IoU of 1.0 — perfect overlap reported for boxes that do not touch.

def iou_bad(a, b):
    x1, y1 = max(a[0],b[0]), max(a[1],b[1])
    x2, y2 = min(a[2],b[2]), min(a[3],b[3])
    inter = (x2-x1) * (y2-y1)                    # no clamp
    aa = (a[2]-a[0])*(a[3]-a[1]); ab = (b[2]-b[0])*(b[3]-b[1])
    return inter / (aa + ab - inter)
print(round(iou_bad((0,0,10,10), (20,20,30,30)), 4))
# 1.0

2. NMS threshold trade-off. Run NMS on three overlapping boxes at thresholds 0.3, 0.5, and 0.9. Report how many survive at each, then describe a scene where the low threshold is wrong and one where the high threshold is wrong.

You should get: two survivors at 0.3 and three at both 0.5 and 0.9; interpret these results using the pairwise overlaps.

# Reuse iou and nms from the main text.
boxes = np.array([[0,0,10,10], [2,2,12,12], [5,5,15,15]])
scores = np.array([0.9, 0.85, 0.8])
for t in (0.3, 0.5, 0.9):
    print(t, nms(boxes, scores, t))
# 0.3 [0, 2]
# 0.5 [0, 1, 2]
# 0.9 [0, 1, 2]

3. Where IoU is flat. Move a predicted 10-by-10 box horizontally relative to a fixed ground-truth box. Compare overlap, edge contact, and strict separation. At contact, estimate the left and right slopes separately. Then explain how GIoU changes the separated case and give a configuration where it still has a flat direction.

At contact, zero IoU does not imply a zero derivative: the two one-sided slopes differ. Strict separation gives a flat neighborhood for IoU.

def giou(a, b):
    a, b = checked_box(a), checked_box(b)
    inter = (max(0., min(a[2], b[2]) - max(a[0], b[0]))
             * max(0., min(a[3], b[3]) - max(a[1], b[1])))
    union = ((a[2]-a[0])*(a[3]-a[1])
             + (b[2]-b[0])*(b[3]-b[1]) - inter)
    enclosing = ((max(a[2],b[2])-min(a[0],b[0]))
                 * (max(a[3],b[3])-min(a[1],b[1])))
    return inter / union - (enclosing - union) / enclosing

truth = (0, 0, 10, 10)
def shifted(x):
    return (x, 0, x + 10, 10)

for x in (5., 10., 20.):
    print(x, round(iou(truth, shifted(x)), 4), round(giou(truth, shifted(x)), 4))
# 5.0 0.3333 0.3333
# 10.0 0.0 0.0
# 20.0 0.0 -0.3333

h = 1e-4
at_contact = iou(truth, shifted(10.))
left = (at_contact - iou(truth, shifted(10. - h))) / h
right = (iou(truth, shifted(10. + h)) - at_contact) / h
print(round(left, 4), round(right, 4))
print(giou(truth, (2,2,4,4)), giou(truth, (3,3,5,5)))
# -0.05 0.0
# 0.04 0.04

References

• Sermanet et al. (2014). OverFeat: Integrated Recognition, Localization and Detection Using Convolutional Networks. ICLR.
• Girshick, Donahue, Darrell, and Malik (2014). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. CVPR.
• Girshick (2015). Fast R-CNN.
• Ren, He, Girshick, and Sun (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. NIPS.
• Liu et al. (2016). SSD: Single Shot MultiBox Detector. ECCV.
• Redmon, Divvala, Girshick, and Farhadi (2016). You Only Look Once: Unified, Real-Time Object Detection. CVPR.
• He, Gkioxari, Dollar, and Girshick (2017). Mask R-CNN.
• Lin, Goyal, Girshick, He, and Dollar (2017). Focal Loss for Dense Object Detection.
• Rezatofighi et al. (2019). Generalized Intersection over Union: A Metric and a Loss for Bounding Box Regression. CVPR.
• Tian, Shen, Chen, and He (2019). FCOS: Fully Convolutional One-Stage Object Detection.
• Redmon and Farhadi (2017). YOLO9000: Better, Faster, Stronger.
• Zhou, Wang, and Krähenbühl (2019). Objects as Points.
• COCO API: detection evaluation implementation.
• Torchvision: non-maximum suppression and its threshold convention.