AI News Feed
Market watch
Large Language Models

Nvidia: The harness, not the AI model, is the real hero for long-horizon tasks

Nvidia research shows a custom harness boosted Claude Opus 5's ARC-AGI-3 score from 30% to 100%.

The researchers created their own souped-up harness called the Agentic Variation Operators (AVO). It is not a new Nvidia product; Nvidia produces open bits and pieces of tech for building harnesses under the Nemo brand. The key addition was a supervising agent that acts like a CEO to nudge the main agent when it goes off direction or explores a dead end.

Adel El Hallack, vice president of product in Nvidia's AI unit, told TechCrunch that an agent is more than just an API of the model. "It is the model. It is the scaffolding around the model, which we call the harness, i.e. the set of tools that it utilizes. It is the runtime and the associated skills and libraries that we give it access to."

Long-horizon tasks require stringing many decisions together, sometimes over days, to produce completed work, in contrast to an AI spitting out a response to a prompt. Getting AI to do such tasks without getting distracted is a holy grail in agentic research. Microsoft published research in April that tested 19 LLMs on long-horizon document editing and found all models filled the documents with errors. Models stringing decisions together have also been caught deleting users' files, even whole databases, or turning to criminal behavior to achieve objectives.

The choice of ARC-AGI-3 is significant because it consists of 2D games with no instructions; the model must figure out how to play and win. A 100% score means the model beats the games as well as humans. OpenAI was so flustered by its models' abysmal scores (less than 10%) that it conducted its own research last month, finding that tweaking two harness settings tripled scores, but none hit 100%.

Nvidia's results add to growing evidence that model choice is far from the only factor in agentic performance. Databricks published research in July showing the harness, more than the model, dramatically impacts AI costs. Databricks CEO Ali Ghodsi told TechCrunch, "You can pick the same model but different harnesses, and you get significantly more cost if you use the wrong harness."

El Hallack emphasized that open harnesses allow users to turn more knobs to drive up accuracy, and called for an open agent stack with control across harness, infrastructure, and runtime. Nvidia believes this is required to usher the ecosystem forward securely.