AI Factory Is Becoming the Computer, Reshaping Semiconductor Race

For the first several years of generative AI, the discussion centered on accelerators. GPUs were scarce, Nvidia became the defining company of the AI infrastructure cycle, and the key questions were how many GPUs could be obtained and how quickly they could be deployed. SiliconANGLE describes that as phase one. The center of gravity is now moving outward from the chip, it says, as the AI factory increasingly operates as a computer.

The analysis points to a broader set of components: central processing units, GPUs and custom XPUs, memory and high-bandwidth memory, scale-up and scale-out networking, chiplets, advanced packaging, optics, cooling, software and electricity. Entire campuses are being designed to operate as enormous computers, which SiliconANGLE says changes the economics, competitive dynamics and geopolitical implications of semiconductors.

SiliconANGLE said it recently heard from leaders including AMD Chief Technology Officer Mark Papermaster, Broadcom semiconductor chief Charlie Kawwas, Samsung Electronics’ Paul Cho and OpenAI hardware leader Richard Ho. Their comments focused on what the semiconductor ecosystem may look like in the coming years. The analysis concluded that AI is forcing the industry to redesign the computer at the same time AI begins redesigning how the computer itself gets built.

The idea that one processor architecture wins everything is increasingly hard to reconcile with AI workloads, according to the analysis. Training, reasoning, inference, retrieval, agents and specialized enterprise workloads have different compute characteristics. Papermaster argued that broad-purpose CPUs and GPUs are not going away, and that modular architectures and chiplets allow suppliers to create workload-specific variations while retaining common foundations. AMD’s multiple Venice designs were cited as an example, including configurations tailored toward emerging agentic workloads.

At the other end are custom accelerators. Kawwas framed XPUs as a market mainly available to a relatively small number of frontier AI companies operating at enormous scale, SiliconANGLE reported. Custom silicon requires enough workload volume, software-stack ownership and predictable demand to amortize development costs. That is why hyperscalers and frontier model companies are moving deeper into silicon: Google with tensor processing units, Amazon Web Services with Trainium and Inferentia, Microsoft with Maia, Meta Platforms continuing to invest in its internal accelerator strategy, and OpenAI pushing its own architecture through Jalapeño with Broadcom.

OpenAI’s rationale, as described by Ho, is full-stack optimization. SiliconANGLE reported that Ho said Jalapeño was optimized around the “kernels, memory movement, networking and serving patterns” that matter for frontier models. The analysis says the frontier of AI infrastructure is moving from buying chips to designing systems around workloads.

Memory is another major change. Historically, architects could design the compute system and then attach an appropriate memory hierarchy. AI flips that assumption, because the volume of parameters and data movement means memory bandwidth, capacity, power consumption and physical proximity to compute increasingly determine system performance. The discussion focused on co-designing logic, memory and packaging, including HBM qualification and 3D-integrated memory, rather than treating them as separate purchasing decisions.

Cho described the shift by saying, “Memory is no longer a supporting player in AI infrastructure, it is becoming the design center.” SiliconANGLE said that is exactly right and added that the AI infrastructure race is no longer simply a compute race; it is becoming a data-movement race. Every time information travels across a board, rack or data center, it costs time and energy, which is driving more aggressive integration of compute, HBM and advanced packaging.