The widely used Geekbench benchmarking software plans to display a cautionary notice whenever Intel's upcoming 'Arrow Lake Refresh' desktop processors activate the company's novel IBOT capability. The reason stems from the benchmark provider's uncertainty regarding the reliability of performance figures obtained with this feature active.
Intel's latest Core Ultra 200S Plus desktop CPUs, referred to as the Arrow Lake Refresh lineup, share many similarities with their 2024 Arrow Lake counterparts. A notable enhancement is the Intel Binary Optimization Tool (IBOT), which optimizes code in select programs for smoother operation and boosted efficiency. This tool is also compatible with the Core Ultra Series 3 processors, known as Panther Lake.
The challenge arises from IBOT's limited documentation, prompting Primate Labs, the creators of Geekbench, to publish a brief blog entry highlighting these concerns.
In practice, running Geekbench on a Core Ultra 200S Plus system or CPU will trigger a provisional alert stating: 'This benchmark result may be invalid due to binary modification tools that can run on this system.'
This alert underscores a fundamental challenge in benchmarking. Users of laptops or desktop computers lack a precise metric for assessing speed, making direct comparisons with other devices difficult. Benchmarks address this by generating standardized numerical values for objective evaluation.
Primate Labs' position involves two main points. First, while the exact mechanics of IBOT remain unclear, the team believes it alters the benchmark's own code. As a result, a Core Ultra 200S Plus processor executes modified instructions compared to those on an AMD chip, per Primate Labs' assessment.
'Since the tool modifies the benchmark, and it is unclear to both Primate Labs and the general public how these changes occur, results generated with the tool are not comparable to results generated without it,' stated Primate Labs.
Additionally, Geekbench lacks the ability to detect whether IBOT is active, introducing further ambiguity. It's possible that reported scores exceed actual performance, but Primate Labs currently has no method to verify this.
As a trusted benchmarking utility, Geekbench's approach is appropriate: Intel released the Core Ultra 200S series without fully explaining IBOT's functionality, leading to outreach to Intel leadership for clarification. This discussion occurred prior to Primate Labs' announcement. Intel has been contacted for a response, which will be shared upon receipt. As suggested by Primate Labs, this may resolve as a short-term clarification issue.
Evaluating game performance tends to be more reliable. In reviews of the Core Ultra 200S Plus, tests were conducted with IBOT enabled and disabled for the limited games it supports, yielding consistent outcomes. However, for certain synthetic benchmarks that could leverage IBOT, validity remains questionable for the time being.
Overall, the narrative around the Core Ultra 200S series holds steady: it offers marginal improvements over the initial Arrow Lake models, trails AMD's Ryzen 9000X3D processors, and comes at a more affordable price point.
Mark has contributed to PCWorld for the past ten years, drawing on three decades of tech reporting expertise. He has produced more than 3,500 pieces for PCWorld, focusing on PC processors, accessories, and Windows, among various subjects. His work has appeared in outlets like PC Magazine, Byte, eWEEK, Popular Science, and Electronic Buyers' News, earning a Jesse H. Neal Award for news coverage. Recently, he cleared out a stockpile of Thunderbolt docks and USB-C hubs due to space constraints in his workspace.