Fastino Releases GLiNER2.5-Decide, a 340M Open-Weight Decision Model That Runs on CPU

GLiNER2.5-Decide is a non-generative classifier. It uses a DeBERTa-v3-large encoder and is fine-tuned from gliner2-large-v1. It produces no generated tokens and requires no prompt template. Label sets are passed at call time. Each question in the schema declares permitted answers and whether it expects one answer, multiple answers, or an ordered value. Schemas can carry instructions, examples, label descriptions, and rules linking answers across questions. The pipeline has two stages: the encoder reads text and schema together and scores every permitted answer, and a constrained decoder searches for the highest-scoring joint assignment allowed by the declared rules. Fastino said the model does not reason, explain, or answer open questions; it is a specialist for operational decisions.

Fastino illustrated joint decoding with a guardrail example. Decoded independently, the model flagged prompt injection at 0.82 and also labeled the same prompt safe at 0.52, producing conflicting outputs even though the attack was detected. Joint decoding applies a rule that any detected harm requires an unsafe verdict. The model then returns safety=unsafe and harm_type=prompt_injection together. Downstream code can use those scores to block, route, or escalate. Schemas can express implications, exclusions, cardinality limits, and ordinal bounds. The same encoder can extract entities, relations, and structured records with character-level offsets in one forward pass, while classification answers do not return evidence spans.

Fastino evaluated the model on Fast Decisions, an internally generated, held-out suite with 5,100 test examples across 17 datasets. The tasks cover customer operations, domain routing in banking, clinical, travel, and benefits, and general content understanding. The metric is exact-match accuracy, where a prediction counts only if its label set matches the reference exactly. GLiNER2.5-Decide averaged 60.1%. By comparison, JevK, a 54B-class Qwen3.5 decoder, scored 57.5%; SemIf, a Qwen3.5-4B decoder, scored 56.4%; GLiFormer large-v1, a single-pass encoder, scored 49.0%; and Laya, a 421M ModernBERT encoder, scored 46.6%. GLiNER2.5-Decide led 9 of the 17 datasets. Intent routing was its strongest area, with 75.3% on support intent and 64.3% on banking intent, ahead of the next-best models by 18.6 and 8.6 points.

Fastino benchmarked the checkpoint end to end at batch 1 with a 2-head, 15-label schema. At 64 tokens, p50 latency was 167.3 ms on a 48-vCPU Intel Xeon Platinum 8581C, 43.6 ms on an NVIDIA T4, 43.4 ms on an NVIDIA L4, 38.3 ms on an NVIDIA V100, and 47.3 ms on an NVIDIA A100. Short requests are dominated by fixed preprocessing and kernel-launch overhead, keeping the GPUs within 9 ms of each other. At 1,024 tokens, the A100 pulls ahead at 52.6 ms, versus 75.6 ms on V100 and 131.4 ms on L4.

In code, a multi-head call adapted from the model card loads AutoExtractor from gliner2 and calls classify_text with a text string and a schema containing intent, urgency, and route labels. Single-label heads return one string, while multi-label heads return every label above a cls_threshold. Labels can carry descriptions, and ordinal scales are passed as ordinary strings such as “0” to “10”. Fastino targets model routing, tool calling, browser and computer use, guardrails, context pruning, LLM-as-a-judge, and simulations. Fine-tuning works locally, fully or with LoRA, through the GLiNER2 trainer. A SKILL.md file exposes the hosted workflow to coding agents.

Fastino also published GLiNER2.5-Decide-1B, built on the Ettin 1B encoder. It scored 59.6% on the same suite, just under the 340M model. For multilingual input, Fastino points to GLiNER2.5-multi-Decide, a 287M model scoring 56.7%. The underlying architecture is described in the GLiNER2 paper.

Editor's Summary

Fastino Labs released GLiNER2.5-Decide, a 340M open-weight decision model under Apache 2.0 that returns structured answers, probabilities, confidence, and feasibility metadata and runs on CPU, GPU, or air-gapped systems. The company reported that joint decoding resolves conflicting outputs, and that the model averaged 60.1% on its Fast Decisions suite while leading 9 of 17 datasets. It also published a 1B variant and a 287M multilingual variant.