CASPA Summit: AI Infrastructure Hardware Becomes a Systems Game

The report said that memory and optical laser capacity are booked out for years, that copper is giving way to light, and that power systems built on cycles of up to a decade must keep pace with chips that turn over every year.

The summit marked the 35th anniversary of the Chinese American Semiconductor Professional Association. The CEO Summit interviews were conducted by theCUBE Research’s John Furrier with executives from Western Digital, Semtech, Rebellions, Multibeam, ScaleFlux and Normal Computing. Irving Tan, CEO of Western Digital, said much of the industry’s focus had been “very much on compute and on memory,” but “our view is actually AI is a data system.” He added: “Ultimately, you need the entire ecosystem to work together and to be able to move in lockstep.”

Tan said data must be split into hot, warm and cold tiers across cloud, edge and devices. Citing IDC, he said the amount of data generated is projected to triple to more than 700 zettabytes over three years, and Western Digital expects data storage demand to grow at a compound annual rate of at least 25% over five years.

Connectivity has become a choke point, according to Hong Hou, president and CEO of Semtech. The company supplies chips inside the optical transceivers and copper cables that link thousands of AI processors into a single computer. Hou said every link flap can force a job to restart, so connectivity governs GPU uptime and revenue. Hyperscalers want more bandwidth at lower power, leaving more of a data center’s fixed power budget for compute.

Inference economics are pushing chipmakers toward workload-aware racks, said Sunghyun Park, co-founder and CEO of Rebellions. Agents now consume tokens around the clock, making performance per dollar per watt the critical metric rather than raw teraflops, he said. Beyond today’s rack-scale systems, Park described racks tailored to whether a workload is latency-critical, throughput-hungry or energy-constrained. Because models change every few months, Rebellions designs flexible hardware and sees the network as the next bottleneck as context windows grow.

David Lam, founder of Lam Research and CEO of Multibeam, said Moore’s Law is reaching its limit and advanced packaging that combines logic, memory and input/output chips is taking over as the industry’s driving force. Multibeam shrank an electron-beam column from 80 centimeters tall to the size of a marking pen, allowing arrays of columns to write a wafer in parallel. Lam said the resulting maskless lithography avoids the reticle limit that forces large quantum and AI chips to be stitched together.

Greg Ernst, chief business officer of ScaleFlux, said the memory wall may not be solved for five years. Even an 8 billion-parameter model serving many users needs far more memory than high-bandwidth memory can hold, said Ernst, who recently joined ScaleFlux from Intel. Compute Express Link lets multiple servers share a pool of memory, and hyperscalers reusing older DDR4 modules in those pools are reviving interest in the technology.

Faris Sbahi, co-founder and CEO of Normal Computing, said new memory fabs take three to four years to come online, leaving most memory from top suppliers sold out through 2028. The startup is moving away from high-bandwidth memory toward commodity LPDDR5 and performs some inference calculations directly in memory to cut cost and energy use.