General Instinct Releases InstinctFlash for Real-Time Robotics Model Serving on Jetson Thor

The post says the largest gain comes from combining those runtime optimizations with a distilled few-step diffusion scheduler. For LingBot-VA, that combination produced a speedup of up to 33.78x by reducing the original 25 visual steps and 50 action steps to 2 visual steps and 4 action steps. Across 50 Robotwin2.0 tasks, with 1,153 episodes evaluated per configuration, LingBot-VA with InstinctFlash at 2 visual / 4 action steps achieved a 90.5% success rate. The baseline at 25 visual / 50 action steps achieved 92.1%, according to the same post.

InstinctFlash currently supports eight VLA and world-action model families, including pi0.5 and NVIDIA Cosmos Policy, across RTX 4090 and 5090 GPUs and Jetson Thor. Guanming wrote that users can provide a fine-tuned checkpoint and the framework handles the rest, exposing the accelerated model through a Python runtime or an OpenPI-compatible WebSocket server.

The project grew out of a deployment problem the team encountered with robot policies: the models were improving, but inference was often too slow for the control loop the team wanted. For pi0.5, mixed-precision GEMMs and CUDA graphs speed up computation and reduce launch overhead. For Cosmos, caching avoids redundant computation across diffusion steps. The denoising step in world-action models depends on the previous one, which the team said motivated its work on few-step distillation.

The framework includes six areas of optimization: CUDA graph capture, memory planning and separation of prefill from repeated execution; reuse of KV and conditioning state across diffusion steps and prediction calls; specialized attention paths for different model architectures; fused operations and kernels tailored to specific backends and tensor layouts; FP8 and mixed-precision execution; and few-step distillation for diffusion and action generation.

Guanming wrote that teams at Samsung, Siemens, and other robotics startups have used InstinctFlash for model acceleration on VLAs, WAMs, and diffusion-based world models. The framework is now being opened up to broader access. The Hacker News post included a link to a YouTube demonstration of a 5B world-action model running in real time on Jetson Thor, along with links to the project's GitHub repository and a blog post with implementation details and benchmarks. At the time of publication, the post had 4 points and no comments.