Neoclouds reshape enterprise infrastructure as AI inference demands surge

Junxia Zhou, senior product manager at Supermicro, said agentic AI is moving into production and inference is becoming the dominant workload. Because inference is highly sensitive to latency, neocloud providers deploy AI closer to end users and require proven, replicable infrastructure configurations across multiple locations. The comments came during an interview with theCUBE, SiliconANGLE Media's livestreaming studio.

The executives argued that neoclouds hold an advantage over hyperscalers because they are purpose-built for AI and carry none of the technical debt of legacy architectures. Vast Data's Phil Manez, vice president of strategic initiatives, said the rise of neoclouds is strong validation that AI cannot run on legacy systems. Hyperscalers have a lot of technical debt, making it hard to turn around, and the path to success in AI is to work with someone who knows how to build purpose-built infrastructure, he said.

Crusoe, an energy-first neocloud, partners with Vast to use its disaggregated shared-everything architecture (DASE) for a high-performance, low-latency storage platform. Crusoe also uses Kioxia's flash memory and SSDs to improve power efficiency. Anders Graham, senior director of SSD marketing and business development at Kioxia, noted that combined demand for flash production and enterprise SSDs has exceeded demand from mobile and client applications for the first time, years ahead of expectations. He attributed the unprecedented demand to neoclouds such as Crusoe, which are building out AI-purpose-built infrastructure.

The panel identified memory and storage as two major constraints on agentic AI. Graham said high-capacity quad-level cell technology will ease bottlenecks, while Zhou saw a shift toward liquid cooling in AI data centers. Zhou said power availability is a challenging issue, as air cooling alone is becoming difficult and expensive at scale with rising GPU power consumption. Liquid cooling improves power efficiency, enables higher rack density and lowers operating costs.

Omar Lari, senior director of products at Crusoe, said lines between compute, storage and infrastructure have blurred, and collaborators must continually reevaluate the right combination of these elements. He described it as a high-stakes game of poker, working with vendors to ensure storage is available at the right time.