Kunlun Wanwei's Skywork AI Releases Matrix-Game 4.0, an Open-Source Game World Model for a Single RTX 5090

The core design is to separate game logic from visual generation. The team's judgment is that instead of letting a video model implicitly remember the world, a game engine should explicitly maintain it. A Coding Agent turns natural language into game code; the engine acts as referee, maintaining world state, executing rules, and producing structural frames; the video model acts as painter, generating final visuals from those frames. In a demo, a shield absorbs damage before health is deducted, health reaching zero ends the game, and rules such as weapon fire rate can be changed in natural language. The same route can turn a time trial into an interception chase.

Technically, Matrix-Game 4.0 is adapted from Wan2.2 5B Fun Control into a video-to-video model. It uses 148-channel concatenated input, a 30-layer causal DiT, dynamic captions from the engine's visible objects and real-time states, spatial memory reference images, a first frame for continuity, and cached recent history. Training data were built in Unreal Engine 5 as paired structural and fine-grained images covering CitySample, open desert, complex castle, and high-dynamic air combat scenes. The runtime uses few-step autoregressive distillation to reduce denoising to one or two steps and generate four frames at a time, along with CUDA Graph, FP8 fused kernels, KV ring caching, parallel attention over KV windows, and the lightweight TAEHV codec. The text encoder, DiT and VAE all stay resident on one RTX 5090. Peak VRAM is about 18.5 GiB; steady-state generation reaches 83 FPS with one-step denoising and about 57 FPS with two-step denoising, while the actual game runs at 60 FPS.

Comparison with other systems is part of the release's claim. QbitAI reported that Nvidia researchers released StreamRender-H3 around the same time, also following the route of code running the world and a video model generating images. Its official deployment uses eight GB200 GPUs; each video block uses two-step denoising, and diffusion rendering takes 218 milliseconds per streaming step. Including video block collection, decoding and playback, end-to-end latency is about 545 milliseconds when two video blocks form one complete feedback cycle. Against that, Matrix-Game 4.0's roughly 112 milliseconds from keypress to display is about one-fifth, with hardware dropping from eight GB200s to one RTX 5090. QbitAI noted that resolution and latency measurement methods are not fully identical, so direct ranking is not appropriate.

Leiphone.com offered a broader comparison. It said Code World Model has about 10.2 seconds end-to-end latency, StreamRender-H3 about 545 milliseconds and AgentGarten about 878 milliseconds, while Matrix-Game 4.0 is 112 milliseconds. It listed compute requirements as 1×H200 141GB, 8×GB200 and 1×H100 respectively, compared with 1×RTX 5090 for Matrix-Game 4.0. On open source, Leiphone.com said Code World Model and StreamRender-H3 are already open, AgentGarten is partially open, and Matrix-Game 4.0 opens code, weights and structural assets together.

The project follows the Matrix-Game line. According to QbitAI, version 1.0 let the generated world understand player input, 2.0 addressed real-time operation, 3.0 added long-term memory, and 3.5 worked on spatial and character stability. Version 4.0 turns to the question of when a real-time generated world becomes a game, and its answer is to put rules in code and the engine. The team has kept the series open source since 1.0. The current 480p output still has room to improve, and the team says it will continue work on a refiner for higher resolution, cross-player view consistency for multiplayer, finer visual control, and support for more game types and scenes.