MemTensor open-sources MemT 1.0, a memory-native model that learns from its own mistakes

The company says the difference is not only that memory has moved off a plug-in and into the base model, but that the model is trained to read, write, update and recall memory by itself, so accumulated experience becomes part of its capability. Most current systems store chat logs and search them, which the company argues cannot turn a correction into a habit.

In MemT 1.0 the model's capacity is split in two. A backbone network carries general knowledge. A separate dynamic memory state holds what a specific user or task has produced so far; states are independent of each other and feed directly into each subsequent generation step. At the end of every turn the memory state is refreshed through a token-level update rather than by retraining the model, with a gating mechanism learned in training deciding what to keep and what to overwrite. A parallel memory attention mechanism retrieves the stored state into the current computation.

The memory state has a fixed size, so continued interaction only refreshes existing matrices. To hold more detail in that fixed volume, the team introduced a Mixture-of-Storage architecture, in which a base mixture-of-experts layer handles computation while several memory experts divide the read and write allocation. Training used 34 billion tokens in two stages: 20 billion tokens of continued pre-training, in which the model predicts future segments from past ones, followed by 14 billion tokens of post-training in which a teacher model scores memory extraction and the error is fed back.

Across 11 memory benchmarks covering long-term retention, dynamic updates and personalisation, with histories from 16K to one million tokens and no access to the original text at test time, MemT-1.0-Flash averaged 50.6 and MemT-1.0-Nano 47.0, ahead of the strongest parametric memory baseline in the same group by 12.5 and 8.9 points. Removing the post-training stage cut Nano's average from 47.0 to 36.3; raising the number of memory experts from two to four added up to 2.7 points.

In one test described in the technical report, a user gave an exhibition's original date, then settled on a Sunday rescheduling while mentioning an unused Monday alternative, after which the pair talked for more than 500 turns and about 66,000 tokens without mentioning the exhibition again. The model reported both the original and revised dates, said the venue, ticket price and contact were unchanged, and separated the Sunday decision from the Monday option. In a 16K-token history it said it did not know the user's birthday rather than inventing one, and in a 33,000-token dialogue it checked badges and registration records to judge which of two people could collect a tool box.

For coding, the team distilled more than a thousand reusable lessons from tens of thousands of successful trajectories and injected them where a model stalled on errors, empty returns or repeated无效 actions — the report says more than 42,000 samples in which the model failed without a lesson and succeeded with one were used to train the memory module. On 500 SWE-bench Verified tasks, Nano with lessons pre-written into memory rose from 66.07% to 69.33% average success, and from 54.00% to 61.60% on passing all three runs of a task; a control group given the same lessons as text in the context scored 68.87% and 61.60%. In an online setting, where the model first works unaided and lessons are written into memory only when it errors or loops three times, success reached 69.73% and the three-run rate 62.80%, against 69.13% and 61.80% for the text-lesson group. On the 300-task, nine-language SWE-bench Multilingual, the same mechanism lifted Nano from 60.00% to 63.56% and from 45.33% to 55.33%, compared with 63.00% and 53.67% for text lessons.

On cost, the company reports that with two GPUs, single concurrency and a batch size of one, Nano runs about 1.20 times faster end to end than a full-context approach at 64K tokens of history, 1.86 times at 128K and 3.99 times at 256K. At 128K, the persistent state needed to restore a single interaction history is about one sixty-sixth the size of a full-context KV cache, excluding shared model weights.

The release places MemT 1.0 against personal-assistant products such as Meta Muse, OpenAI Dots and xAI Grok Bot, which rely on external retrieval, and follows MemTensor's earlier Memory³ research and MemOS system work. The code is published on GitHub under MemTensor/MemT-1.0, with model weights on Hugging Face.