Embodied AI Preprints: Gemini Leads as Robot Skill Judge, 2B S²-VLA Outperforms Larger Models

The first study, VLM-AR3L, benchmarked VLMs on ten embodied tasks across four simulation suites, including Cart Pole, rope straightening, moving a cup of water, soccer, drawer opening and four Minecraft chores. The models were shown two first-person frames and had to decide which frame was closer to completing an instruction. Gemini 2.0 Flash was the only tested model to score above 70 percent on every task, reaching 91 percent on Cart Pole and 84-87 percent on MetaWorld manipulation tasks. GPT-4.1-nano mostly stayed between 60 and 66 percent, and CLIP-style contrastive models struggled, with MineCLIP scoring 39-59 percent even on Minecraft tasks. Some small open-source models were strong in specific scenes: Phi-3.5-Vision hit 90 percent on milking and shearing, DeepSeek-VL2-Tiny got 85 percent on combat spider, and MiniCPM-o-2.6 reached 73 percent on water-state judgment.

The same team found that directly using such VLMs as real-time reward providers for reinforcement learning is impractical. They estimated that hundreds of thousands of API calls could raise a single training run to tens of thousands of dollars and introduce latency that would slow learning many times over. Absolute single-frame scoring also drifts during training and cannot handle cyclic tasks; the report gives an example where an agent trained with absolute reward on a ring world never learns because no global ordering exists on a closed loop. VLM-AR3L therefore combines an absolute reward, for global direction toward the goal, with a relative reward that compares the current observation to one from k steps earlier, in order to sense per-step progress. A bidirectional confidence check awards positive reward only when the relative model is consistently confident in both directions. To cut cost, the large VLMs are queried offline on sampled observations to create preference labels, and a small twin network is trained locally to produce rewards during policy learning. This cut VLM API queries to about one-twentieth of the original count. In Minecraft tasks, the method achieved 85 percent success on combat spider and 95 percent on milking, and it completed long-horizon workflows in which human-engineered sparse rewards yielded zero success.

The second study, S²-VLA, addresses cumulative error propagation in long-horizon manipulation. The authors argue that static fusion of visual, linguistic and action-history features leaves attention proportions fixed during an entire task, so a small early positioning error is not corrected and can compound into a failure after roughly ten steps. Their model uses a lightweight recurrent network to maintain a belief state that estimates task progress and deviations without manual stage labels. The belief state generates dynamic gate weights for three parallel attention channels: local visual cross-attention for fine spatial alignment, global intent cross-attention for sub-task planning, and action self-attention for trajectory continuity. In a 31-step basket-loading example, the visual channel weight rises during precise aiming, the intent channel weight rises during grasping and object switching, and action self-attention dominates smooth movement phases.

With only 2 billion parameters, S²-VLA reached 96.4 percent success on the LIBERO-Long benchmark, exceeding most 7-billion and 8.5-billion parameter VLA models. The model requires about 7GB of GPU memory during inference and runs at 80.8Hz, while typical 7B VLA models run at only a few hertz. The authors say such efficiency is important for real deployment on household or factory devices.

Together, the two preprints point toward a practical division of labor: large foundation models can act as offline supervisors rather than low-level controllers, while small local models execute real-time actions. S²-VLA demonstrates that for long physical interaction sequences, adaptively scheduling attention may matter as much as or more than parameter count alone.