Ant AI Security Lab Releases SingProbe to Detect Risks During LLM Generation

Large models are moving into daily question answering, office assistance and customer service. Users expect AI to provide useful answers quickly, and they also need those answers to be safe and reliable. The challenge is to detect unsafe content and fabricated information in time while reducing the impact of safety review on response speed and user experience, according to the lab.

At present, large model applications usually rely on independent external guardrail models to review inputs or outputs. This approach is general and direct, but it requires processing content separately, adding computing and deployment costs. If a full answer is checked only after it is generated, risky content may already have been shown to users. If the checking frequency is increased, the operating burden rises further, the lab said.

SingProbe seeks to make safety judgments run alongside model generation. It reuses internal information already produced during LLM inference and uses a lightweight safety detection module to continuously output risk scores. This means that while the model is generating an answer, it can also signal whether the current content carries safety or factual reliability risks, giving the system time to issue warnings, stop generation or retry. For example, the lab said, users asking an AI about health issues may worry about inappropriate medication advice, and users searching for information with AI need to guard against fabricated references or facts. In these cases, SingProbe can provide risk signals during answer generation, helping applications take timely measures.

According to tests in the production environment of the Ling-3.0-flash model, SingProbe introduced less than 0.5% extra overhead during the decoding stage. The lab's evaluation showed that the flash version surpassed selected public baselines on answer safety classification and streaming safety detection tasks, and was roughly on par with the reference baseline on hallucination detection. The lab described this as a new technical path that balances detection capability and operational efficiency.

For streaming generation scenarios, the lab also released an evaluation benchmark called SingStreamBench. The benchmark focuses on the specific moment when a model shifts from a normal answer to risky content. It not only tests whether a guardrail can find the risk, but also examines whether it triggers too early and whether detection is timely, aiming to measure safety protection in a way that is closer to real applications.

In medical generation scenarios, the research team further explored technical application from risk identification to on-demand correction and proposed SingProbe-Med. This system continuously monitors medical risks during generation and, only after trigger conditions are met, applies decoding intervention to locally high-risk content. According to the team's medical evaluation on AntAngelMed-100B, full intervention corrected 25.03% of answers that the baseline model originally got wrong, showing the potential of endogenous risk signals for safety control during generation.

SingProbe has been adapted to 29 mainstream open-source large models, including the Ling-3.0 series, GLM-5.2/5.3, Qwen and DeepSeekV4, and has been integrated with the SGLang and vLLM inference frameworks. The related code, models and evaluation benchmark have been open-sourced. The team said it will gradually expand support for more open-source models and push safety protection more tightly into large model inference and deployment processes.