YOLO

Two-stage object detectors looked at every image region separately, slowly. YOLO — “You Only Look Once” — predicted every bounding box in one forward pass over a grid. Real-time detection became possible, and stayed possible.

§ 00 · DETECTION VS CLASSIFICATIONTwo different vision problems

Classification is “what’s in this image?” — one label per image. Detection is the harder problem: “what objects are in this image, where are they, and what’s their bounding box?”

For each detected object you need three things: the class (person, car, dog), the bounding box (4 numbers: x, y, width, height), and a confidence score. An image can contain zero objects or twenty.

§ 01 · ONE FORWARD PASS, ONE GRIDDetection as a regression problem

The original 2016 YOLOYOLO. You Only Look Once. A 2016 object detector by Joseph Redmon et al. that frames detection as a single regression problem over a grid, in one forward pass — dramatically faster than the prevailing two-stage detectors. framed detection as one big regression problem. Divide the image into a grid (originally 7×7). For each cell, the network predicts:

• Whether an object’s center falls in that cell.
• The bounding box for that object (x, y offset from cell, width, height as fraction of image).
• Class probabilities.

All of this comes out of one forward pass through a CNN. Versus previous approaches that proposed thousands of candidate regions and classified each separately, YOLO got near-state-of-the-art accuracy at 45 frames per second on a GPU — a 30× speedup. The trade-off was slightly lower accuracy on small objects. Real-time vision became practical.

§ 02 · ANCHOR BOXES & CLASS SCORESTemplates for object shape

One cell can have multiple objects centered nearby; objects come in very different aspect ratios (a person is tall, a car is wide). Version 2 of YOLO introduced anchor boxesanchor boxes. Pre-defined box shapes (typically 3–9) that each grid cell predicts adjustments to. Different anchors handle different aspect ratios (tall, wide, square), helping the network specialize per shape. — pre-defined box-shape templates the model learns to adjust. Each cell predicts several boxes, one per anchor, each with its own confidence and class scores.

The anchors are usually picked by clustering the bounding boxes in the training set (k-means on widths and heights). Three to nine anchors per cell is typical. With anchors, the network can specialize: “tall thin anchor” learns to detect people, “wide anchor” learns to detect cars.

§ 03 · NMS — PICKING WHICH BOXES TO KEEPSuppress duplicate predictions

Adjacent cells will often produce overlapping predictions for the same object. The raw output of YOLO is a noisy list of boxes, many of which are near-duplicates with slightly different scores.

Non-Maximum SuppressionNon-Maximum Suppression. A post-processing step that removes duplicate detections. Iteratively: take the highest-scoring box, mark it kept, suppress any other box whose IoU (intersection-over-union) with it is above a threshold. Repeat with the next-highest. Standard for any modern detector. (NMS) cleans this up. The algorithm:

1. Take the highest-confidence box. Keep it.
2. Find any other box whose IoU (intersection over union) with the kept box is > threshold (e.g. 0.5). Discard those.
3. Pick the next-highest remaining box. Repeat.
4. Stop when no boxes remain.

The result is a clean per-object list — one box per object, no duplicates. NMS is its own little algorithm and exists in basically every detection pipeline ever built, YOLO included.

§ 04 · REAL-TIME, REAL-WORLD, ONGOINGThe YOLO line continues

YOLO has gone through many versions since 2016 — v3, v4, v5, v6, v7, v8, v9, v10, and beyond. Different teams; sometimes confusing branding. The core recipe has been remarkably stable: backbone (feature extractor) → neck (multi-scale feature fusion) → head (predict boxes + classes per cell). The improvements have been incremental:

• Backbone evolution — Darknet → CSP-Darknet → EfficientNet → Transformer hybrids.
• Multi-scale prediction — predict at three resolutions (coarse, mid, fine), so small and large objects each get attention.
• Anchor-free variants (since YOLOv6) — drop anchors entirely, predict (cx, cy, w, h) directly. Simpler, comparable quality.
• Better label assignment — which ground-truth object should each prediction be responsible for during training? The art of label-assignment heuristics keeps improving.

Production uses: autonomous driving, robotics, sports analytics, retail (people counting, basket analytics), agriculture (crop monitoring), public-safety surveillance, and increasingly inside multimodal LLMs as the visual front-end.

§ 05 · TAKING THIS FORWARDWhere detection is heading

Three directions worth following:

• Transformer-based detectors. DETR, RT-DETR, YOLO-World — replacing or augmenting the CNN backbone with attention. Catching up on speed; benefit from text conditioning.
• Open-vocabulary detection. Models that can detect objects described in text at inference time, not just a fixed class list. Bridges detection with vision-language understanding.
• Segment-anything-style. SAM treats detection almost as a side-effect of segmentation. The line between detection and segmentation is blurring.

§ · GOING DEEPERAnchor boxes, NMS, and how YOLO evolved

Redmon’s 2015 YOLO paper made object detection a single forward pass: divide the image into a grid, each cell predicts bounding boxes and class probabilities. Earlier two-stage detectors (R-CNN family) proposed regions first, classified them second — much more accurate, much too slow for real-time. YOLO traded some accuracy for the ability to run at 45 FPS, which made it the practical choice for video and embedded deployment.

Each version added a refinement. v2 (2016) introduced anchor boxes — predefined shape priors that handle aspect ratios. v3 (2018) used multi-scale prediction. v4–v9 are increasingly engineering-driven: data augmentation tricks (mosaic, mix-up), better backbones, training-time regularization. The conceptual successor is DETR (Carion et al. 2020), which replaced the grid with a transformer and bipartite matching — slower but more flexible. Modern production detection is a mix of YOLO descendants (for speed) and DETR descendants (for accuracy and instance-aware tasks).