Caterpillar and CoreWeave seek to shorten physical AI learning loop for construction

Demand for new data centers, power plants and highways is fueling a construction boom. Caterpillar has years of experience with autonomous equipment in mining. Extending that technology to construction requires systems that can adapt to more variable conditions, according to Brandon Hootman, vice president of physical AI platforms and construction autonomy at Caterpillar.

“Once that mine site gets instantiated, it does change, but it doesn’t change frequently,” Hootman said. “You go to construction as an industry. Polar opposite of mining. If you think about all of the dynamics of what has to happen, taking a structured system and matching it to an unstructured environment, it’s really, really challenging to do.”

Hootman and Richard Ahlfeld, senior vice president of physical AI at CoreWeave, spoke with theCUBE Research’s Dave Vellante and John Furrier at the Fully Connected event. They discussed why physical AI reshapes the infrastructure behind autonomy and how the two companies are shortening the learning loop for construction machines.

AI clouds were first built to train foundation models and then tuned for agentic inference at scale, according to the executives. Training an autonomous excavator means ingesting telemetry and vision data, simulating a digging scenario a million times and layering reinforcement learning on top. CoreWeave recently launched a Physical AI Field Engineering service that embeds its engineers with customers’ domain experts, Ahlfeld said.

“Physical AI now is an entirely different beast,” Ahlfeld said. “That requires, first of all, a lot of storage. It requires a different infrastructure.”

Caterpillar’s digital ecosystem already holds about 18 petabytes of federated data from machines, dealers and customers, Hootman said. Training autonomous equipment also requires perception data synchronized with machine control and performance data.

“That 18 petabytes that we’ve collected so far is just a drop in the bucket to the amount of data that it actually takes to go train physical AI to operate in a world like a construction site,” he said. “But you take [Light Detection and Ranging] data, camera data, that multi-second control data and performance data coming back in, you’re talking about terabytes of data within a given day for just one machine.”

CoreWeave named Caterpillar among its enterprise customers in its second-quarter results announcement. Caterpillar began working with CoreWeave this year, drawn by both graphics processing unit capacity and applied expertise, according to Hootman. Working with Nvidia, the partners use AI models to annotate and label incoming field data.

“Now you’re taking things that were months to maybe weeks,” Hootman said. “And now you’re getting it down to hours and your feedback loop between that happening and being able to use it in your simulation environment, or your training environment, happens within the given workday.”