PrismML launches Bonsai 2 27B, a 5.9 GB ternary model for local AI
PrismML launched Bonsai 2 27B, a 5.9 GB ternary multimodal model based on Qwen3.8 27B that runs on consumer hardware and retains 98.2% of its average benchmark performance, according to the company.
The release arrives two months after the first Bonsai 27B, whose ternary variant retained about 95%, according to MarkTechPost. The new model keeps the Qwen3.8 27B architecture unchanged and has 27.36 billion parameters: a 24.35 billion-parameter language backbone, 2.54 billion parameters in embeddings and the language-model head, and a 0.47 billion-parameter vision tower. The backbone uses hybrid attention with about 75% linear-attention layers and 25% full-attention layers.
PrismML uses ternary weights, which take one of three values: -1, 0 or +1. In full-size models, weights are usually represented with 16 bits. Every group of 128 ternary weights shares one FP16 scale, and PrismML applies a blockwise Hadamard rotation with block size 1,024 before assigning ternary values; the runtime applies the matching transform to activations before each multiply. The company does not publish how it assigns the ternary values, according to MarkTechPost.
Ternary weights cover embeddings, attention projections, MLP projections and the LM head. Only 26.2 million parameters, or 0.0976% of the model, stay in higher precision: the recurrent state path and normalization weights. In GGUF format, the vision tower ships separately as a 0.63 GB file and is loaded only for image input. The model accepts text and images and supports a 262,000-token context.
PrismML evaluated the models in thinking mode using EvalScope and vLLM on H100 GPUs. MarkTechPost reported that Bonsai 2 scored 83.9 on average across 20 benchmarks, against 85.4 for Qwen3.8 27B in FP16. Knowledge and reasoning retained 96.9%, math 99.5%, coding 99.3%, agentic and tool calling 97.3%, instruction following 101.7%, and vision 96.3%. SiliconANGLE reported that Bonsai 2 and Qwen3.8 were within 3 points on agentic and tool calling, at 77.6 and 79.8, with coding aggregates of 81.6 and 82.2 and knowledge and reasoning scores of 82.7 and 81.3.
The average retention hides larger losses on long-horizon agent tasks. Bonsai 2 scored 52.8 on Terminal-Bench 2.1, against 69.7 for Qwen3.8 27B, and 60.8 on SWE-bench Verified, against 80.6, about 75% retention. At medium reasoning effort, the model averaged 79.3, against 82.6 for the FP16 baseline; low effort is not supported. MarkTechPost said all results are PrismML's own and have not been independently reproduced.
Against conventional quantization, MarkTechPost reported that an IQ2_XXS build of Qwen3.8 27B averaged 75.2 at 7.3 GB. It scored 78.6 on AIME26 while Bonsai 2 scored 95.83, and 70.05 on LiveCodeBench v6 while Bonsai 2 scored 90.07.
PrismML reported batch-size-one decode speeds on its custom kernels, measured September 16, 2026. An RTX 5090 reached 142.5 tokens per second at 0.582 milliwatt-hours per token; an RTX 4090 reached 96.7 with PTQ1_0; a 72-watt L4 reached 32.1. On Apple laptops, an M5 Max reached 46.8 tokens per second and an M5 Pro reached 27.7. SiliconANGLE reported 46.8 tokens per second on Apple's M5 Max and said the model consumes extremely low power per token, making it 40% more energy-efficient than other 8B models running at full precision.
The model weights are available under the Apache 2.0 license. GGUF files require PrismML's fork of llama.cpp because stock llama.cpp rejects the PTQ1_0 and PQ2_0 types, according to MarkTechPost. The supported path is the Bonsai-demo repository, where users run ./setup.sh and then ./scripts/start_llama_server.sh for chat, vision and tools at localhost:8080. Mac users can use the MLX pack with its bundled loader, and a WebGPU demo runs the model in a browser.
The two reports differ on the size of the full-precision baseline. SiliconANGLE said Qwen3.8 weighs around 56 GB at its full 16-bit uncompressed size and that Bonsai 2 reduces it to around 5.9 GB. MarkTechPost put the FP16 baseline at 53.80 GB and Bonsai 2 at 5.93 GB. Both reports said the compressed model is intended to let users run inference locally, avoiding the latency, cost and third-party data sharing associated with cloud inference.
SiliconANGLE reported that simple translation, summarization and search organization could run on device, while long-horizon task comprehension and research might still require cloud models. For enterprises, local models could handle simple tasks involving sensitive information, while more complex work scales to cloud-based models.