Computer Vision (CV)

Computer vision extracts useful information from images and video: what is present, where it is, which pixels belong together, and how a scene changes over time. The input records light through a sensor; the desired output describes something about the world. That gap makes viewpoint, lighting, scale, occlusion, and the task definition central to a vision system.

The output determines the problem

Imagine a factory inspecting a product. Classification could predict whether an image contains a defect. Detection could locate each defect with a bounding box and a category. Semantic segmentation could identify every defect pixel; instance segmentation could separate two distinct scratches even when they share a category. Tracking would connect the same object across frames. An image-level label cannot by itself tell a detector where the defect is.

These output contracts affect both annotation and evaluation. A classification error concerns an image label, while a detector must get both the category and location right and handle missed or duplicate detections. Box overlap is often summarized by intersection over union: the shared area divided by the area covered by either box. Segmentation compares pixel regions, where a small boundary error may matter differently from missing a tiny defect entirely. Labeling effort depends on objects, boundaries, annotation tools, and required expertise; there is no universal cost ranking that makes every classification task cheap.

From pixels to useful representations

A digital color image is typically a grid with several channel values at each position. Decoding, resizing, color ordering, and normalization establish the numeric input seen by the model. If training uses one channel order and serving another, identical model weights receive different evidence. Resizing can also erase a small object before any model has a chance to detect it.

Traditional pipelines can use engineered measurements such as edges, local descriptors, or geometric features. Learned models fit representations from data instead. The ImageNet work by Krizhevsky and colleagues was a prominent demonstration of large convolutional networks, rather than the invention of all learned visual features. Classical geometry and image processing remain relevant where their assumptions match the task, sometimes alongside learned components.

What convolution contributes

A convolutional layer applies the same small set of weights at many spatial positions. For a single-channel image and a 3 × 3 filter, each response is a weighted sum of nine nearby pixels plus a bias. Sliding that detector creates a feature map. Different filters can respond to different patterns, and later layers combine responses across larger regions. The model learns the weights; naming a filter an edge detector is an interpretation to check, not a rule enforced for every channel.

For 32 filters on one input channel, a biased 3 × 3 layer has 32 × (9 + 1) = 320 parameters regardless of image width. A dense mapping from a 24 × 24 single-channel image to 32 outputs has 576 × 32 + 32 = 18,464. These layers produce different output shapes, so this is a comparison of parameter sharing, not equivalent predictors or equal compute. With three input channels, each ordinary filter spans all three and the convolutional count becomes 32 × (27 + 1) = 896.

Away from boundaries, a stride-one shared filter shifts its responses when the input shifts. This translation equivariance is different from invariance, where the output stays unchanged. Strides, padding, pooling, and the final readout can alter the relationship. A classifier may seek tolerance to small translations, while a detector must retain location. Neither convolution nor augmentation guarantees that every transformed image gets a correct result.

Vision transformers instead can represent image patches as tokens and use attention to exchange information among them, with positional information preserving layout. Their spatial assumptions differ from convolution. Pretraining, architecture, augmentation, and compute determine the practical tradeoff; a claim about one model trained at one scale does not establish a universal winner. A feature extractor and a task-specific output head together determine what the system can predict.

Building evidence for deployment

A pretrained encoder can provide reusable features while a smaller head is fitted to the new labels. Freezing it lowers the trainable parameter count, but may leave features that cannot distinguish the target detail. Fine-tuning changes the representation at additional training cost and can overfit a small dataset. Compare against a simple baseline, and inspect errors before deciding that model size is the limiting factor.

Augmentation should represent changes that preserve the intended label. Cropping can remove the defect, horizontal reflection can reverse text, and aggressive color changes can destroy evidence needed for a color-sensitive task. Labels such as boxes and masks must transform consistently with the image. This is why an augmentation recipe should follow the task rather than being copied as a generic way to create more data.

Split evaluation data according to the unit that must generalize. Neighboring video frames, near-duplicate photographs, or images from the same product can leak information across a random split. Hold out the relevant subjects, sites, devices, or periods, and also inspect performance under anticipated lighting, resolution, and population changes. A low aggregate error can conceal failures on rare objects or specific groups. Architecture, labeling, training, and deployment conditions can all contribute; poor data coverage is one possible cause, not the only explanation.

Connect model outputs to the decision: missed defects and unnecessary manual inspections have different costs. Choose thresholds and review paths using representative validation data, then evaluate the complete image-processing and prediction pipeline. Measure latency at the required resolution and frame rate as well as quality. The learning machinery is introduced in neural networks; image classification in practice develops transfer learning, augmentation, and evaluation for that specific output type.

References

• Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90. doi.org
• PyTorch documentation, torch.nn.Conv2d. docs.pytorch.org
• NumPy documentation, numpy.roll. numpy.org