TechRadar column: Measure AI by business outcomes, not token or adoption vanity metrics

The column used a NASCAR analogy to describe how measurement can distort behavior: if leadership decides the winner is the car that burns the most fuel, crews would tune engines to waste fuel and drivers would race while braking, producing record consumption and a last-place finish. It said wasteful consumption metrics drove AI measurement programs for part of the year, with companies setting up internal leaderboards that ranked employees by how many tokens they burned. A Meta employee built one such leaderboard, crowned the top performers "token legends," and quietly took it down after the numbers leaked externally, according to the column. By spring the trend had a name, tokenmaxxing, it said.

By the start of summer, the backlash arrived, the column said. It cited Uber as acknowledging that it had already spent its entire AI budget for the year in a matter of months. More recently, the column said, a swing toward using the minimum number of AI tokens, or tokenminning, leads to the same problem: if leadership defines winning as using the least fuel, the winning car never leaves the starting line.

Burning or saving tokens should not be the end goal, the column argued. The objective is shipping more products at higher quality with the same or fewer people, and the job of a leader is to define the most strategic path to that finish line and build metrics that support it, it said.

The column described vanity metrics as numbers that look good on a slide, leaderboard or dashboard but do not change how a business is run. In AI, it said, even highly technical and complex metrics can become vanity metrics. It gave examples including GPU utilization rates that boast about how much raw compute an organization has hoarded regardless of whether it is solving real problems, and AI features shipped per quarter when leadership adds a text summarizer to every legacy product to pad release notes.

Such metrics can feel rigorous but are detached from the work itself, the column said. When leaders do not relate metrics to end goals, engineers may race toward the nearest countable item, it added. The column pointed to the gap between work completed and the right work being completed: leadership may celebrate 95% adoption of GitHub Copilot because licenses are deployed, mistaking superficial access for deep engagement and assuming productivity is rising. Developers might generate more code faster, but if the tool is not deeply integrated into a refined process, pull request sizes inflate and the bottleneck shifts to human code review, the column said. Cycle time stays stagnant or bug rates rise because humans are rubber-stamping AI hallucinations, according to the column.

To close that disconnect, the column said companies should tie depth of use directly to outcomes and business goals. If an organization has 95% adoption but cycle time has not dropped by 30% or rework has not decreased, it does not have high use, the column argued; it has an expensive new software distribution.

The column said a single number rarely tells a story. Producing twice as much code but taking twice as long to ship does not represent efficiency, it said; it creates more work. The right question is not how much AI is being used but whether the work is actually getting better, according to the column.

For engineering teams with established workflows, the column recommended starting with the outcome they want to move and then using a framework such as DORA or SPACE to check whether they got there. It described the framework as the instrument, not the goal. DORA tracks the health of a delivery pipeline, including how often a team deploys, how long changes take, how quickly it recovers from failures and how often changes cause incidents. SPACE tracks developer productivity across five dimensions, including outcomes and satisfaction. Neither was designed for AI, the column said, which is why they are useful: they can help measure whether AI made the work better, not whether AI got used.

The column recommended pairing one of those frameworks against AI tool usage and business outcomes such as cycle time, rework rate and cost per change. A 25% increase in AI adoption that correlates with a drop in delivery stability is not a win, it said. The column cited the 2025 DORA report as finding that many teams got exactly that result when they deployed AI into underprepared systems: the tools amplified what was already there, making good engineering foundations better and fragile ones more fragile.

For roles outside engineering, including marketing, finance and operations, the column recommended building from first principles. Teams should start with one outcome they are already trying to move, such as response time, pipeline conversion or forecast accuracy, establish the finish line before AI enters the workflow and measure toward the same outcome afterward. If the outcome moved, teams should determine whether AI caused it and whether the result can be repeated; if it did not move, they have still learned something, the column said.

Editor's Summary

The TechRadar column argues that AI programs risk chasing token usage and adoption statistics instead of measurable delivery and business results. It cites tokenmaxxing leaderboards and Uber's budget overrun as examples, and recommends tying AI use to frameworks such as DORA and SPACE and to specific outcomes. It also says 2025 DORA data show AI can amplify weak engineering foundations.