77 Object Detection and Localization
Classification, the task covered throughout this book so far, answers “what is this?” for an image or a patch as a whole. Object detection answers a more specific pair of questions at once: what is present in this image, and exactly where. The “where” part is called localization, typically expressed as a bounding box, the smallest rectangle that encloses the object or region of interest. A field-monitoring system built on drone imagery genuinely needs both: not just “this image contains a stressed region” but “the stressed region is located here, in this specific part of the field,” information that turns a general alert into an actionable one.
77.1 Sliding-Window Detection and IoU
One of the earliest and most direct approaches to object detection is the sliding window: a fixed-size window is moved systematically across every position in an image, and some scoring function is evaluated at each position to decide how likely that window is to contain the object of interest. The window position with the strongest score becomes the detected bounding box. Modern detectors such as YOLO (Joseph Redmon et al., 2016) replace this exhaustive scan with a single, far more efficient pass, but the underlying idea, score many candidate regions and keep the best one, is the same.
Once a bounding box has been predicted, its accuracy is measured against a known true box using Intersection over Union (IoU) (Paul Jaccard, 1912):
\[ \text{IoU} = \frac{\text{Area of Overlap}}{\text{Area of Union}} \]
An IoU of 1 means the predicted and true boxes match exactly; an IoU of 0 means they do not overlap at all. In practice, a detection is often considered correct if its IoU against the true box exceeds some threshold, commonly 0.5, a single number that captures both how well the object was found and how precisely it was located.
77.2 Worked Example
A synthetic 20-by-20 grid representing a drone’s NDVI scan of a field section, with a background NDVI around 0.75 and a rectangular 5-by-5 patch of stressed vegetation (NDVI around 0.25) embedded somewhere inside it. The task: find that patch’s location using a sliding window, without being told in advance where it is, and check the result against the true, known location.
77.3 Object Detection in R
77.4 Reading the Result
The detected bounding box should match the true one closely, or exactly, here, since the stressed patch’s NDVI values are sharply different from the background and the sliding window is exhaustive: it checks every possible position, so it cannot miss the strongest candidate. An IoU close to 1 confirms both parts of the detection task succeeded together, the patch was found, and it was found in the right place, not just somewhere in its general vicinity. This exhaustive sliding-window approach does not scale well in practice: scanning every position at every possible window size across a full-resolution drone image is far too slow for real-time use, which is exactly the computational problem architectures like YOLO were built to solve, replacing the window-by-window scan with a single forward pass through a convolutional network that predicts bounding boxes and class scores simultaneously across the whole image at once.
Summary
| Concept | Description |
|---|---|
| Foundations | |
| Object Detection | Answers both what is present in an image and exactly where, unlike classification alone |
| Localization and Bounding Boxes | Localization expresses an object's position as a bounding box, the smallest enclosing rectangle |
| Sliding-Window Detection | Scores every candidate window position across an image and keeps the position with the strongest score |
| Intersection over Union (IoU) | Overlap area divided by union area between a predicted and a true bounding box, from 0 to 1 |
| Modern Detectors: A Single Efficient Pass | YOLO and similar architectures replace an exhaustive scan with one forward pass predicting boxes directly |
| Worked Example | |
| Worked Example: Locating a Hidden Stressed Patch | A 20x20 synthetic NDVI grid with a hidden 5x5 stressed patch, located without knowing its position in advance |
| Implementing a Sliding-Window Scan in Base R | A double loop scores every valid window position by mean NDVI, keeping the lowest-scoring (most stressed) one |
| Evaluating the Detection with IoU | IoU against the known true patch location quantifies both whether and how precisely the patch was found |