AI Infrastructure Shifts to Agent-Scale Design

According to Leiphone, OpenAI said Jalapeño achieved 1.5 to 1.9 times higher AI work per watt at peak throughput positions on GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T, along with 1.7 to 3.6 times lower end-to-end latency. In the high-interactivity range, the improvement was 2.1 to 4.1 times. The chip has a rated power of 700W, but sustained power stayed below 550W on the tested workloads. Reddit commenters compared Jalapeño with NVIDIA Rubin without reaching a clear ranking. Some said Jalapeño reaches Rubin's efficiency band on a per-dollar token basis and does not rely on speculative decoding; others countered that the public tests used relatively regular workloads, Rubin's software optimization path is different, and Jalapeño still lacks public data on complex agent long-context loads. At Hot Chips 2026, NVIDIA showed Groq 3 LPU taking over low-latency decode in Vera Rubin systems, while Google split its TPU 8 into a training-focused TPU 8t and an inference-focused TPU 8i, with the latter carrying more HBM and SRAM. These designs all point to a common bottleneck: decode-stage memory bandwidth and data movement rather than raw compute.

Anthropic, meanwhile, extended the +50% weekly quota boost for Claude Code to Aug. 31; it had originally been set to expire Aug. 19. Leiphone reported that Hacker News users discussed how quickly quotas drop even for relatively simple tasks. The analysis said Claude Code consumes far more than the final code patch: reading files, searching call chains, running tests and processing logs all pull tokens into the context, and longer tasks make the system more dependent on compaction and memory. Anthropic recommends using sub-agents to isolate high-output tasks and notes that prompt caching lowers repeated computation, but caching does not decide whether old context should continue to exist. The report also warned that agent-written code may leave future agents unaware of the reasoning behind previous commits, creating a new kind of "legacy code mud pile."

In code hosting, Cursor began rolling out Origin early beta to all paid plans around the same time as GitHub's major outage on Aug. 17. Origin unifies repositories, PRs, checks, reviews, merges and Automations under one system, aiming to build code hosting for "agent scale." At a June Origin demo, a single repository showed 22.6 commits per second, though Leiphone notes this was a live demo figure, not an independently reproduced production benchmark. A July study of 33,596 agent PRs across 2,807 repositories found that 40.2% of repositories had overlapping agent PRs; in sampled replays, text merge conflict rates reached 41.7% for cross-agent PRs and 19.8% for concurrent PRs from the same agent. Cursor's acquisition of Graphite is also tied to stacked PRs and merge queues. Origin allows teams to first mirror a GitHub repository, then migrate agent workflows, and eventually detach from GitHub.

The Sora versus Codex comparison further illustrates how different AI loads consume compute. According to Leiphone, OpenAI CEO Sam Altman said on a podcast that Sora is a good product and could be a good business, but it is too compute-intensive, and Codex took higher priority at the same time. Leiphone's analysis argues that Sora's compute is continuous and exclusive: each diffusion round processes an entire spatiotemporal latent grid, making it hard to reuse history through KV caches. Codex, by contrast, breaks a task into multiple inference and tool-call phases, allowing the scheduler to move GPU work between sequences during tool waits. This difference in load structure, rather than the absolute amount of compute, helps explain why the two products expand at different speeds.

Taken together, these moves show the AI industry shifting from model capability to inference-system efficiency. Leiphone's critique of Anthropic's technical stack identifies five bottlenecks: machine-readable markers hurting code generation, adaptive thinking changing model tiers, long-context state consistency, compaction fidelity, and agents self-inflicting errors. NVIDIA and Google's chip adjustments, along with Cursor's Origin, are all re-designing infrastructure for the operating rhythm of agents.