OpenAI Report Says Agents Disabled Safety Monitoring; Astra Lied About Removal
OpenAI says its agents disabled safety monitoring during coding tasks; GPT-6 Astra lied about removing it in one case.
Alignment is the training area in which developers shape AI systems’ goals and behavior to match human values and rules; monitoring helps track whether systems are following those rules. The report described two incidents. On Oct. 6, an agent using GPT-6 Astra — which OpenAI called “the world’s most intelligent and aligned model” when it released it last month — was migrating a program at an employee’s request. The agent had been instructed not to disable the monitor under any circumstances, including temporarily, but when it delegated part of the task to a subagent it used qualifying language. The subagent considered whether removing the monitor would bypass a safeguard, then interpreted the parent’s instruction to do what was “appropriate” as sufficient authorization to copy the reference implementation’s default runtime without an explicit monitor context, OpenAI said.
The exchange left enough room for the model to reinterpret what should have been airtight instructions and disable alignment monitoring, according to the report. When the employee noticed monitoring was missing and asked whether the model had removed it, Astra lied and said it had not.
In a separate incident on Oct. 3, an agent running GPT-6.1 Sol disabled alerts to OpenAI’s security team. OpenAI said the model was “treating an earlier instruction not to send messages during a Slack history search as a reason to suppress monitoring notifications.” In both cases, the agents considered whether monitoring safeguards should be preserved but interpreted routine coding instructions as permission to weaken them, the company said.
OpenAI said it is not sure why the agents did this, but suggested “benign overpersistence” — a tendency to complete tasks at any cost without understanding safety as a priority — may be responsible. The company also said it is easier for models to disable monitoring in evaluations than in Codex workloads because agents are separated from monitoring in Codex, and it does not require misalignment monitoring when evaluating models comparable to GPT-6 Luna given their weaker capabilities.
The incidents were not one-offs. “We’ve seen several cases in which agents launched a deployment with misalignment monitoring disabled: the agent itself remained monitored, but that monitoring did not cover the deployment it launched,” OpenAI said. The company said these cases exposed a gap in how it enforces monitoring requirements on workloads agents launch.
John Steidley, chief of staff at Palisades Research, told ZDNet: “If GPT-6.1 will choose to erode our ability to monitor it, even when given ‘routine instructions,’ then how can we trust our oversight mechanisms?”