Enterprise IT Is Not Built for AI Inference, TechRadar Analysis Warns

The analysis points to rising AI investment and inference demand. IDC estimated global AI investment surpassed $300 billion in 2025. Deloitte found inference already accounts for about two-thirds of compute, while McKinsey projected it will represent 30% to 40% of total data center demand by 2030. Despite those figures, the analysis said dedicated infrastructure for inference has not kept pace.

Training and inference impose nearly opposite requirements, according to the analysis. AI training has relied on massive, centralized GPU clusters with extreme power density and specialized high-speed interconnects designed to maximize communication within a single data center room. Inference is defined by continuous information flow and unpredictable user demand rather than a predefined workload. Once prompts arrive, the analysis said, enterprises need geographically distributed compute that responds to and scales with real-time demand.

Because inference runs around the clock and requires millisecond-fast responses for end users, it depends on commodity Ethernet, geographically distributed edge sites and low-latency network connections rather than isolated throughput inside one data center. The design choices that make an enterprise data center optimal for training work against running inference at scale, the analysis said. The cost structure also differs: training is described as a one-time capital event, while inference is a recurring operational cost that scales with every user, query, agent interaction and more.

Autonomous AI agents are adding further strain. The analysis said agents do not generate a single response and stop; they plan, execute across multiple steps, call external tools, coordinate other agents and pass outputs between systems. That multiplies data traffic across networks. It cites Gartner's prediction that by the end of 2026, 40% of enterprise applications will be integrated with task-specific agents, and IDC's projection that agent usage will surge tenfold by 2027, with token and API call volumes multiplying by a factor of thousands.

The analysis also cites McKinsey's finding that most enterprise WANs are currently unable to support AI data traffic patterns, and that AI now ranks as the second-highest infrastructure investment priority. Burst traffic is another problem. Agentic workloads are not a steady flow; a single orchestration event can trigger dozens of downstream API calls simultaneously. Networks sized for average loads can hit capacity ceilings when agents coordinate at scale, creating latency and reliability problems that lead enterprise AI pilots to fail before reaching production, according to the analysis.

To adapt, the analysis argues enterprises must build inference-forward architecture from the ground up, starting with hardware. CPUs are critical for planning, scheduling, tool-call coordination and the multi-step reasoning loops that define agentic workloads. The industry's heavy focus on the GPU market during the AI training boom has diverted attention from the need for CPUs in AI inference workloads. As inference workloads increase in scale, CPUs move from a supporting role to a critical one, acting as the main filter to the orchestration layer.

When viewed through an agentic lens, high-core-count server CPUs can handle the workload more economically than GPU accelerators, the analysis said. But one cannot function without the other: inference still demands GPU function. The analysis's broader warning is that enterprises optimizing spending for training may find they have the wrong foundation for the inference era.