Comet Cloud targets small businesses as GPU cloud demand shifts to inference

The GPU cloud market has been shaped by its largest buyers, which can commit to multiyear contracts for scarce accelerators. Demand for neocloud capacity is shifting as AI startups prioritize burst capacity, or extra computing power for short spikes in demand. Kato said hyperscalers once rented GPU capacity by the hour, day or week. “You go to [Amazon Web Services] now; they want a multiyear contract from you. They’re picking who they’re taking,” he said. “To the small and medium businesses, they’re not even coming up on that scope at all to be able to have that AI. Really that’s our focus.”

Comet Cloud’s route to AI infrastructure began in payments. Kato’s credit card processing business led him into Bitcoin mining, which he came to view as an arbitrage of electricity, a cost that also matters in AI computing. The company now offers dedicated clusters, a model aimed at customers such as medical offices and law firms that need their data isolated from other customers. “What we do is we put together clusters and we’re giving clusters to a whole company. All their information, all their security, everything is within that cluster. There’s no one else tied or connected to it,” he said.

For now, Comet Cloud supplies bare-metal infrastructure to partners such as Andromeda, a compute marketplace that recently standardized storage across more than 50 GPU providers using Weka NeuralMesh. Kato said Comet Cloud plans to move further up the software stack and turn its sites into token factories. Price is central to the pitch, particularly for Nvidia Corp.’s B300 systems. “[Hyperscalers are] going to probably charge you $15, $20 an hour, maybe more. You’re going to come to someone like us and it’s down to $4 an hour. The difference is it’s astronomical,” he said.

Lower prices become more important as inference overtakes training, Kato said, because small businesses will need tokens for everyday tasks such as marketing and customer service. Through its payments merchants and affiliates, including one that works with 50,000 oncology offices, Comet Cloud can deliver third-party AI software to small businesses along with the compute to run it, he said. Most of those businesses do not yet use AI beyond consumer chatbots, according to Kato. “As everyone’s developing these [AI tools], it’s going to make them more efficient, more profitable … When that’s developed and we can offer to them, that’s where it’s really going to change everything,” he added.

Building that capacity requires power, GPU allocation and capital. Comet Cloud has 300 acres under contract on the West Coast and is beginning a relationship with Dell Technologies Inc., whose AI Factory business has reached an inflection point, Kato said. Financing is another challenge: a single 48-rack cluster of Nvidia’s next-generation Vera Rubin systems can require $1 billion to $1.2 billion in contracted revenue to secure financing, according to Kato. “We need the price of tokens to go so low that everybody could afford to use it for their businesses and use those tokens,” he said. “That’s where we should end up.”

Editor's Summary

Comet Cloud is positioning dedicated GPU clusters for small and midsize businesses as hyperscalers favor multiyear contracts. The company says lower hourly pricing and inference demand could bring AI to medical, legal and other small-business customers, while capacity and financing costs will determine how far it can scale.