ACE Robotics and HKU Release StreamPI, Adding Temporal Understanding to VLA Models
ACE Robotics and the University of Hong Kong unveiled StreamPI, a method that gives vision-language-action models continuous temporal modeling, boosting real-robot task success by up to 33 points.
The work is positioned as a step toward what the researchers call continuous physical intelligence. While recent VLA models like π0.5 can handle long-horizon household tasks, they often still depend on the current observation only. StreamPI introduces a streaming multimodal temporal modeling scheme that lets a VLA remember past states, track changes, and use cross-frame information for finer spatial judgments. The key design is to treat each moment as a paired visual observation and task instruction, so the language goal remains anchored throughout. Historical information is cached in a key-value store, allowing the model to call up past representations without recomputing all prior frames.
StreamPI does not add extra parameters or a separate video encoder. Instead, it reworks the attention mechanism of the existing VLA backbone, inheriting the visual-language representations of π0.5. A random-interval streaming training strategy also helps the model cope with irregular real-world timing, where camera capture, inference, communication, and actuator delays can distort the apparent frame rate. The strategy randomly varies time gaps between observations and hides some early information, making the model learn stable temporal relationships rather than frame-rate-specific patterns.
In standardized benchmarks, StreamPI improved LIBERO average success from 96.4% to 97.5% with three input frames, and from 97.0% to 98.3% with five frames. On the long-context LIBERO-Long split, it rose from π0.5's 92.4% to 95.0%. On CALVIN, the average continuous task length reached 4.547, exceeding π0.5's 4.313 and MemoryVLA's 4.090; at the fifth step, success was 85.0% versus π0.5's 79.5%.
In four real-robot tasks, each trained on 100 human demonstrations, StreamPI outperformed π0.5 on tasks requiring temporal memory and fine spatial perception. Narrow-bottle pen insertion went from 40.0% to 66.7%, and paper-cup-in-holder insertion from 60.0% to 92.0%. The gains came not from better object detection, but from understanding what happened before and predicting where objects are heading. The team plans to explore efficient training over more than 100 frames and adaptive cache pruning for even longer and more complex real-world tasks.