Study: Agentic Memory Works Best When Dosed Per Model Capability

The researchers introduced a method called ALTK-Evolve, which lets an agent learn from its own past trajectories by distilling reusable behavioral guidelines and injecting them back at inference time, without updating any model weights. Across the eight models, they identified three patterns: strong models with headroom benefit from the full set of mined guidelines, smaller or weaker models perform best with a compact high-confidence core plus task-relevant retrieval, and already-saturated models show no measurable gain.

For example, DeepSeek-V3.2 (671B MoE) saw a +9.5 percentage point increase in task completion when given its full self-mined guideline set, while gpt-oss-120b (117B MoE) gained +16.1 percentage points with a selective approach. The full guideline set for gpt-oss-120b produced smaller gains and cost about 50% more tokens. GLM-5 (745B MoE) showed no measurable improvement, a pattern the researchers label as saturated, acknowledging that the cause could be the model already near its ceiling or the guidelines not addressing its remaining failures.

The study defines agentic memory not as replaying past transcripts but as a set of guidelines distilled from an agent's own successful and unsuccessful trajectories. The process includes extracting behavioral guidelines, consolidating them into a reusable set, and at inference time providing either the full set or a task-relevant selection. Since no weights are updated, the approach is cheap and portable across models.

Evaluations were conducted on AppWorld, which includes 585 multi-step tasks across 9 simulated apps. Tasks were scored by task goal completion (TGC) and the stricter scenario goal completion (SGC). The researchers note that what determines a model's pattern is not simply parameter count; benchmark headroom, context-window size, architecture, guideline quality, and task distribution all appear to play a role, with separating those factors left as ongoing work. The practical takeaway is that the right dose of memory must be calibrated to each model.