Unapproved AI Use in Healthcare: How to Build a Defensible Response

A practical response framework for healthcare privacy and compliance leaders

When an employee enters patient information into an unapproved AI tool, the response should begin immediately: contain further use, preserve evidence, determine exactly what was submitted and how the service handled it, assess applicable obligations, and document the decision. A policy violation can take seconds. Establishing the privacy impact and a defensible response can take much longer.

That challenge surfaced at two Washington state mental health facilities, where unapproved use of a personal ChatGPT account potentially exposed protected health information belonging to 16 patients, according to Becker’s Behavioral Health. The information may have included names, diagnosis details, and medications. The state found no evidence of further disclosure or identity theft, but that evidence alone does not resolve the compliance assessment.

For healthcare privacy and compliance leaders, the practical question is not only whether policy prohibited the use. It is whether the organization can quickly establish what happened, evaluate the risk to patient information, coordinate the right response, and preserve evidence of a reasoned decision.

Why unapproved AI use can become a healthcare privacy incident

Unapproved AI use often begins with an ordinary workflow: drafting documentation, summarizing a case, organizing information, or reducing administrative burden. The employee may be trying to work efficiently. But when patient information enters an unreviewed service or a personal account, it can move outside the organization’s approved technical, contractual, and governance controls.

In mental healthcare, the potential impact is especially personal. Clinical information can reveal diagnoses, medications, treatment history, and circumstances that patients reasonably expect their providers to protect. The investigation should consider both how identifiable the information is and how sensitive what it reveals is.

Evidence that information was not further disclosed or used for identity theft matters. It is one part of the fact pattern, not a substitute for the required assessment.

How should healthcare organizations respond to unapproved AI use?

A repeatable response should connect containment, fact-finding, regulatory assessment, documentation, and workflow remediation. The exact team will vary, but privacy, security, compliance, legal, and clinical leaders need a shared process and clear ownership.

Contain the activity and preserve evidence

Stop further unauthorized submissions and follow approved evidence-preservation procedures. Coordinate account changes, prompt deletion, access restriction, or device collection so that mitigation does not unintentionally destroy information needed for the investigation. Record each action, when it occurred, who approved it, and what the action could not confirm.

Establish what data entered which service

Identify the employee, task, service, account type, model or feature, dates, prompts, outputs, patient information involved, and affected individuals. Determine the relevant settings and terms, including retention, model training, provider access, connected applications, and deletion behavior. Collect available logs, exports, screenshots, provider responses, and related records through approved procedures.

The product name alone is not enough. A consumer account, an enterprise account, and an embedded feature may have different contractual terms, controls, and data-handling practices.

Assess obligations and document the rationale

Evaluate the event under applicable federal and state requirements, contracts, organizational policy, and other duties. Assign responsibility for the decision and any notifications. Document the facts relied on, unresolved questions, analysis, approvals, mitigation, and final outcome so the organization can show how it reached its conclusion.

Correct the workflow that led to the incident

Ask what the employee was trying to accomplish and why the approved environment did not meet the need. The corrective action may include clearer training, stronger controls, an approved alternative, a simpler approval path, or changes to the underlying workflow. This turns the investigation into a practical improvement rather than a policy reminder alone.

How does the HIPAA breach assessment apply to AI tools?

HHS Breach Notification Rule guidance states that an impermissible use or disclosure of protected health information is generally presumed to be a breach unless the covered entity or business associate demonstrates, through a risk assessment, a low probability that the information was compromised, or an applicable exception applies.

The assessment must consider at least four factors:

1. The nature and extent of the protected health information involved, including identifiers and the likelihood of re-identification.
2. The unauthorized person who used the information or received the disclosure.
3. Whether the unauthorized person actually acquired or viewed the protected health information.
4. The extent to which the risk to the protected health information was mitigated.

For an AI-related event, applying those factors requires facts about the specific service and account. Teams may need to determine whether prompts were stored, whether the provider or another party could access them, whether the information could be used for service improvement or model training, how deletion operates, and what evidence supports each conclusion.

Some facts may remain unavailable. Document that uncertainty rather than treating an assumption as fact. HHS also requires regulated entities to demonstrate that they made required notifications or that notification was not required, making a complete record of the assessment and decision essential.

What makes an AI service appropriate for patient information?

Approval should apply to a defined service, account type, configuration, use case, and data-handling arrangement, not to a product name in the abstract. Healthcare organizations should evaluate the service’s role, contractual terms, access controls, retention, permitted uses, integrations, security, and fit for the intended workflow.

HHS cloud-computing guidance explains that a covered entity or business associate may use a cloud service to process electronic protected health information when it enters into an appropriate business associate agreement with the cloud service provider and otherwise complies with the HIPAA Rules. HHS also states that the organization should understand the specific cloud environment, conduct its own risk analysis, and establish risk-management policies. A signed agreement is important, but it does not replace those responsibilities.

Employees need equally specific instructions: which tools and account types are approved, what information they may enter, which features or integrations are permitted, and how to request help when an approved workflow doesn’t meet the need.

How RadarFirst supports a consistent and defensible response

RadarFirst AI Incident Management gives teams a structured way to investigate AI incidents, assess potential impacts, coordinate response, and document decisions and actions. Teams can capture AI-specific context, bring privacy and other stakeholders into one response process, apply regulatory intelligence, and maintain a traceable record of the rationale and outcome.

For healthcare organizations, that structure connects the AI investigation with the privacy and compliance work it may trigger. People remain accountable for the decision; the process helps them act more consistently, with greater continuity, and with proof of diligence.

Turn a policy exception into a defensible response

Patient trust depends on more than a policy that says what employees should not do. It also depends on how quickly the organization can establish the facts, assess the significance, coordinate action, and explain why it reached its decision.

See how RadarFirst helps healthcare privacy and compliance teams investigate AI incidents, coordinate response, and preserve a defensible record of decisions and actions.