Corvus ISR’s latest public tracker benchmark offers a rare glimpse into the performance of wide-area motion imagery (WAMI) exploitation models using a completely synthetic scene. This approach leverages perfect ground truth data generated from a fixed-seed environment, ensuring that the evaluation is both precise and reproducible across different models. The benchmark compares two tracker versions: a simple greedy nearest-neighbour baseline (v1) and an advanced confirmed-track auction model (v2), providing clear insights into their respective capabilities under identical conditions.
Prime made for students and young adults
- Fast, free delivery for dorm and study essentials
- Prime Video and Amazon Music included
- Member-only deals
The synthetic scene setup is meticulously designed: seed 1337 guarantees deterministic scene generation, with 20 seconds reserved for warm-up followed by 120 seconds of measured data. The sensor model, detection generation, and metric definitions are identical and byte-for-byte the same, with the only difference being the tracker algorithm. This methodology ensures that the only variable affecting performance metrics is the tracker model itself, making the results exceptionally valuable for scientific comparison and validation.
Among the headline results, the v2 tracker reduces the number of ID switches per minute significantly—from 2,042 to 1,183 in the baseline scenario with 150 movers at 2fps, representing a 42.1% improvement. The dense scenario with 400 movers shows a similar reduction, from 14,032 to 8,040, a 42.7% decrease. These figures highlight the substantial impact of advanced association techniques like velocity-consistency gating and noise-scaled reservation prices integrated into v2, which outperform the simpler baseline in challenging conditions.
Notably, the benchmark explicitly publishes failure numbers—not just successes—because synthetic scenes provide perfect ground truth. By doing so, Corvus ISR emphasizes that every future tracker must demonstrate performance against the same deterministic dataset. This stance promotes an honest, measurement-driven approach, contrasting with marketing that often only showcases successes. The detailed data reveals that even the more advanced model still commits thousands of identity errors per minute under stress, illustrating the ongoing challenge of reliable tracking.
From an engineering perspective, v2 runs efficiently, averaging about 1.2 milliseconds per sensor tick at a density of 400 objects—well within real-time constraints. In fact, the entire benchmark is reproducible directly in a browser, as shown on the live demo page. Without any signup or NDA, users can simply press “Run benchmark” to generate identical results, fostering transparency and technical validation.
The significance of this approach extends beyond marketing; it underscores the importance of methodology in performance evaluation using synthetic data with perfect ground truth. The fixed-seed matrix ensures that results are not only objective but also verifiable and repeatable by anyone interested. This transparent benchmarking serves as a foundation for future developments in WAMI tracking technology, where progress can be objectively measured against a consistent standard.
If you’re eager to see how your own algorithms perform or simply wish to understand the state of the art, you can check the public benchmark and reproduce it live. The full suite of results invites educators, researchers, and developers alike to run the benchmark themselves and contribute to advancing the field with transparent, measurable data.
As an affiliate, we earn on qualifying purchases.
synthetic scene simulation tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
