Mirendil Releases Fern 0.1, a 10M-Token Coding Model

Mirendil said the 10-million-token window is equivalent to about 23 MB of code, or roughly 500,000 lines, a codebase on the scale of Megatron-LM held in context at once. Fern 0.1 ranges from 0.1 trillion to 0.5 trillion parameters; the exact count and number of active parameters were not disclosed. It was built from Mirendil-Base after a short round of post-training. No open weights or quantized builds were announced.

The target uses listed by Mirendil include working across larger codebases, agentic coding that runs commands until tests pass, research code such as training loops, data pipelines and evaluations, debugging training runs from long logs and stack traces, reading full agent or training traces end to end, and auditing pretraining shards for duplicates, contamination and junk.

Mirendil said one researcher pretrained Mirendil-Base in 3.5 months, with total compute under $10 million including experiments, agent token spend and data processing. AI agents wrote more than 99% of the code, while the researcher chose what to try and when to intervene. Training data came only from public sources including GitHub, public code, web pages and public datasets. Mirendil said it used no distillation, synthetic data or licensed data, and wrote the code from scratch on PyTorch and JAX.

Fern is the smallest of three planned models. Mirendil said Mirendil-Base-Large, with 0.5 trillion to 2 trillion parameters, is over 80% through pretraining, while Mirendil-Base-XL, with 2 trillion to 10 trillion parameters, is over 20% through.

On long-context retrieval, Mirendil reported that the base model scored 100% on RULER single-needle retrieval at every length from 32K to 10 million tokens. It scored 100%, 98%, 98% and 90% on multi-key retrieval across the same lengths, and 100%, 100%, 100% and 100% on multi-key all-needles. Variable tracking scored 100%, 95%, 75% and 100%, while multi-hop QA on HotpotQA fell from 65% at 32K to 45% at 10 million tokens. Mirendil said the 10-million-token cells used only 20 examples, compared with 100 elsewhere, and that broader long-context results are coming.

Mirendil also tested next-token prediction on its own repositories concatenated into a 42.9 MB stream, measuring cross-entropy in bits per byte, where lower is better. Fern's loss fell from 0.246 at a 32K-byte window to 0.155 at 16 million and 23 million bytes. Other models were scored up to about 1 million tokens, their advertised limit. At a 1 million-byte window, Kimi-K3 scored 0.140 and Fern 0.165. According to the report, Fern is not the best predictor at equal context, but its curve keeps going past where the others stop.

On agentic coding benchmarks, Fern 0.1 trailed the frontier and large open models cited in Mirendil's comparison. It scored 6.1% on Terminal Bench 4.0, versus 12.6% for Kimi-K3, 41.9% for GLM-5.3 and 66.4% for Claude Opus 5.5. It scored 44.2% on DeepSWE v1.1, 53.2% on SWE Atlas QnA, 76.5% on SWE-Pro v2 Hard and 73.4% on SWE-Bench Verified. Mirendil said Fern 0.1 beat Inkling and Nemotron 3 Ultra on four of five tests and tied Beam on SWE-Pro v2 Hard at 76.5%, but trailed Beam on SWE-Bench Verified, 73.4% versus 81.6%. Every frontier and large open model in the table led it.

In the comparison table, Fern 0.1 lists 0.1 trillion to 0.5 trillion total parameters, undisclosed active parameters and up to 10 million tokens of context, with text and code input. Its license was not disclosed and its weights were not released; access is by early-access request. Kimi-K3 is listed at 2.8 trillion total parameters and 104 billion active parameters, with a context of 1,048,576 tokens and text, image and video input; GLM-5.3 is listed at 753 billion total parameters and about 40 billion active parameters. Both are listed with open weights and APIs. At 1 million bytes, the bits-per-byte figures in Mirendil's charts were 0.165 for Fern, 0.140 for Kimi-K3 and 0.170 for GLM-5.3.