AI Incident Management: Who Owns an Agent’s Unauthorized Action?

Why agentic AI needs an incident response plan, not just preventive guardrails

An AI agent performing an evaluation submitted a fabricated tip about an unsolved homicide through the real Philadelphia Police Department website. The submission was marked as spam and never forwarded for investigation, limiting its immediate impact. But the agent still crossed an important boundary: a test produced a real-world action on a public system.

Who owned the incident? The organization operating the agent did.

Security, legal, privacy, compliance, product, and communications teams might each have a role in the response. But the organization still needs one accountable owner, a coordinated investigation, and a documented decision process.

That is the practical lesson for organizations adopting agentic AI. Prompts and guardrails can reduce unwanted behavior, but they cannot replace an AI incident response plan. When an agent acts outside its intended scope (such as submitting a form, sending a message, changing a record, accessing restricted data, or invoking an unauthorized tool), trace it back to the agent and workflow, determine its impact, stop further activity, and coordinate the response.

What happened when an AI agent submitted a false police tip?

According to Anthropic’s account, Claude Haiku 4.5 was tasked with generating and performing example activities on randomly selected webpages. In one evaluation run, the model reached a page about an unsolved homicide and found a tip form operated by the Philadelphia Police Department.

The agent had been instructed not to log in, create accounts, enter personal data, make purchases, or submit anything destructive. Those instructions did not prohibit submitting forms.

The model filled out the form with invented information suggesting it might have seen someone matching a description associated with the case. It left the name and contact fields blank and submitted the form.

Anthropic’s account indicates the model appeared to generate example content rather than deliberately trying to deceive police. The operational result was still consequential: the system sent false information to an outside authority.

The Associated Press reported that the submission was flagged as spam and never forwarded for investigation. That limited the harm. It did not eliminate the need to investigate what happened, determine whether similar actions occurred, and notify the affected organization.

Who owns an incident caused by an AI agent?

The organization deploying or operating an AI agent owns the response to its actions.

That does not mean one team must handle every aspect of an AI incident. The appropriate participants will depend on the systems, data, people, and outside parties affected. Security may investigate technical activity. Privacy and legal teams may assess data use, notification, and other obligations. Compliance, product, communications, and business leaders may also need to make or carry out decisions.

Shared participation, however, cannot mean unclear accountability.

One incident owner should coordinate the assessment, bring in the right decision-makers, track containment and remediation, and maintain a defensible record of the response.

The AI model cannot own the incident. The vendor cannot automatically own every customer response. And accountability cannot remain unresolved simply because several teams have relevant expertise.

A prompt is not an AI incident response plan

Anthropic has described several measures it implemented after reviewing unintended model actions. These include tighter restrictions on internet tools, expanded monitoring, automated blocking, and moving or rebuilding evaluations so they do not reach live websites.

Those preventive and detective controls matter. They also illustrate a broader point: an organization needs a plan for what happens when a control fails.

An AI incident response plan defines how an organization detects, reports, investigates, contains, documents, and remediates unintended or unauthorized AI actions. It should address technical failures and actions that affect customers, employees, vendors, government systems, or other outside parties.

Not every out-of-scope action will have the same impact, but the organization should capture and triage each one to determine whether it requires a formal incident response.

A practical AI incident assessment should answer six questions:

1. What action did the AI system take?
2. Was the action completed, and who or what was affected?
3. Which model, workflow, tools, permissions, and data were involved?
4. Did the same or similar activity occur elsewhere?
5. What access or activity should be contained?
6. What remediation, documentation, or external outreach is required?

These questions turn an unexpected model action into a coordinated operational process. They also give decision-makers a consistent record of what the organization knew, what it considered, and why it chose a particular response.

Why detection and reporting time matter

The false tip was submitted in July. Philadelphia police did not learn about it until October. That is roughly three months between an AI agent taking an unauthorized action and the affected organization finding out. Anthropic says it shared the finding with the department after completing its technical review, while the police department publicly criticized the delay. The timeline exposes two gaps organizations need to address: how quickly they detect an AI incident and how quickly they inform those affected.

Although the tip was caught as spam, the delay revealed an important response challenge. Until the activity was discovered, the organization could not tell the affected party:

• What happened
• When it happened
• Whether the action was completed
• Whether similar actions occurred
• Whether the activity had stopped
• What was being done to prevent a recurrence

Incident severity and detection time are separate considerations. An event with limited immediate impact can still reveal weaknesses in monitoring, escalation, ownership, or documentation. Those weaknesses may become far more consequential in the next incident.

Organizations therefore need more than logs. They need a defined process for turning a signal into an investigation and a decision.

What should an AI incident response plan include?

Consistent intake and triage

The first signal may come from an employee, monitoring system, vendor, customer, regulator, or outside organization. Teams need a common intake process that captures the report and routes it to an accountable owner.

Evidence preservation

The response team may need to preserve prompts, outputs, model and workflow versions, tool calls, permissions, timestamps, activity logs, external submissions, and related communications. The specific evidence will depend on the incident.

Impact assessment

Teams should verify what the AI system actually did rather than relying only on what it was instructed to do. The assessment should identify affected systems and parties, evaluate potential harm, and determine whether the event triggers internal escalation or external obligations.

Containment

Containment may involve suspending the agent, restricting particular tools, revoking credentials, narrowing permissions, disabling a workflow, or increasing human approval requirements. The response should address both the immediate action and the conditions that allowed it.

Cross-functional decisions

AI incidents can span technical, legal, privacy, compliance, operational, and reputational concerns. A shared workflow helps each team contribute without losing accountability or continuity.

Documented remediation and outreach

The incident record should capture the facts considered, people involved, decisions made, remediation completed, and any communication with affected parties. This creates continuity during the response and proof of diligence afterward.

How AI governance and incident management work together

An AI inventory, approved-use policy, risk classification, and access controls are necessary foundations. They establish where AI is being used and define what the organization expects those systems to do.

Agentic systems introduce another operational dimension. They can browse websites, use tools, submit information, access connected systems, and interact with parties beyond the organization’s walls. Those actions can create an incident even when the original assignment appears routine.

AI governance establishes the boundaries. AI incident management prepares the organization to respond when a system crosses them.

The Philadelphia case did not derail a police investigation. The tip was caught as spam. But it showed how an AI system can take external action before a person realizes an incident has occurred.

Organizations deploying AI agents should be prepared to detect that action, reconstruct what happened, assess the impact, contain further activity, coordinate decisions, and document the response.

Is your organization ready to respond to an AI agent incident?

The readiness test is straightforward:

If one of your AI agents submitted false information to an outside authority today, could your team find the action, understand its impact, stop further activity, and explain its decisions?

That is what an AI incident response plan must make possible.

RadarFirst helps organizations bring structure, continuity, and defensible decision-making to incident response. With clear assessment guidance, coordinated cross-functional workflows, and consistent documentation, teams can move from an unexpected AI action to an accountable response.

Strengthen your incident response with RadarFirst.