Hugging Face Releases Carbon-X, a 14.8B Open DNA Model That Reads Sequence and Answers in English

Carbon-X is a decoder-only model initialised from Qwen3-14B, and its central design choice is that DNA and text occupy one token stream inside one decoder. Earlier systems such as BioReason paired a separate DNA encoder with a language model; Carbon-X removes that split. The mechanism is a hybrid tokenizer that keeps Qwen3's byte-pair vocabulary for text and adds 5,515 DNA tokens, made up of 4,096 non-overlapping 6-mers, 1,364 tail tokens and three markers. Each DNA token covers six base pairs, and runs of six or more A, C, G or T bases are tagged automatically. The backbone is a standard Qwen3 stack with 40 layers, a hidden size of 5,120 and grouped-query attention using eight key-value heads, with a 40,960-token context.

The model predicts whether mutations are benign or pathogenic, flags regulatory regions, classifies organisms and predicts the next base in a sequence. It is dense rather than mixture-of-experts and, per the release, runs on a single 80 GB GPU in bfloat16 through vLLM or Transformers. Official GGUF builds are listed at 15.8 GB for Q8_0 and 9.6 GB for Q4_K_M.

Training ran in four stages described on the model card. An embedding warmup covered about 10 billion tokens with the transformer frozen so that only the new DNA embeddings learned. Mid-training used 302 billion tokens at a sequence length of 8,192, mixed as 55% raw DNA, 20% English, 15% biology text and 10% instruction data. Supervised fine-tuning ran for three epochs over 20 dataset subsets, with responses checked by deterministic verifiers built with domain experts. Reinforcement learning used asynchronous GRPO with verifiable rewards over two rounds, run with TRL and vLLM.

Hugging Face reported that the learning rate was the most difficult setting. High rates let the model learn DNA but degraded its English; a lower rate, a 14B backbone and a long run resolved the trade-off. The company also said that applying reinforcement learning directly to Qwen3-14B never beat the 50% chance baseline on FrontierVEP.

All benchmark scores are self-reported by Hugging Face, with Carbon-X run with thinking disabled and competitors run at their best reasoning effort. Carbon-X averaged 66.7% across five DNA benchmarks, against 67.5% for GPT-6.1 Sol (Max), 60.9% for Opus 5.5 (Max) and 50.4% for Kimi K3 (Low). It scored 74.9% on BioReason Coding, ahead of every model tested, and 39.4% on NextBase; it trailed on BioReason Non-SNV at 85.1% against 91.7% for GPT-6.1 Sol and 94.8% for Opus 5.5, and on FrontierVEP at 70.4% against 74.9% and 76.2%. Its weakest result was 29.8% on GenomeQA splice-site questions. Hugging Face put Carbon-X 27 points above its Qwen3-14B starting point and 0.8 points below GPT-6.1 Sol on the average.

The reported margins are shaped by refusals, according to the same account: on NextBase, Opus 5.5 fell from 17.7% at low effort to 1.1% at maximum effort as refusals increased, and Gemini 4 Argon dropped from 58% to 38% on FrontierVEP as effort rose. Carbon-X retained general skills, scoring 79.3% prompt-level strict accuracy on IFEval.