# AI for incident reporting: what it can and cannot safely do

> AI can safely draft incident reports: ask follow-up questions, structure the story, suggest categories and summarize records. It should not decide severity, causes, blame or reporting duties. Keep a person in charge, label AI-drafted fields until approved, log every change, and sign a BAA when patient information is involved.

Source: https://incidentkit.ai/guides/ai-for-incident-reporting · Updated Oct 5, 2026

## The short answer: AI drafts, people decide

AI is useful where the task is language: turning a messy account into a structured record, spotting what is missing, and summarizing long text. It is unsafe where the task is judgment or accountability: severity, cause, fault, deadlines and closing an action.

The rule is simple. The AI drafts. A named person reviews, edits and signs. The record shows both steps. AAAHC's AI governance guide for ambulatory care (August 2026) calls this meaningful human oversight. See [how AI intake changes incident reporting](https://incidentkit.ai/blog/how-ai-intake-changes-incident-reporting).

## AI can ask, fill and flag at intake

| Task | AI role | Person's role |
| --- | --- | --- |
| Ask follow-up questions | Asks what a risk manager would: time, place, who was told, equipment, outcome | Answers and approves the narrative |
| Fill form fields | Proposes values, marked as drafts | Confirms each one |
| Suggest type and routing | Suggests a type. Routing rules, not the model, decide who is notified | Corrects the type |
| Flag gaps and contradictions | Highlights missing times or conflicting details | Resolves them |
| Transcribe or translate | Drafts text from speech or another language | Checks names, doses and times |
| Link related reports | Suggests duplicates or related events | Confirms the link |

Keep the reporter's own words as the source of truth. Everything the AI adds sits beside them, never replaces them.

### A worked example (illustrative)

1. **The reporter writes** "Found a resident on the floor beside the bed around 2 a.m. He said he was going to the bathroom. I didn't see an injury."
2. **The assistant asks** What time exactly? Was the bed alarm on? Who found the resident and checked for injury? Who was told, and when?
3. **The draft fills in** Fields take only what was said, such as "about 2 a.m." and "no injury seen," each marked draft. Unanswered fields stay blank, not guessed.
4. **The reviewer edits** After the nurse's assessment, the reviewer changes the injury field to a skin tear on the left forearm and records the notifications.
5. **The draft stops short of a cause** If nobody said the alarm failed, the draft does not say so. Causes come later, from the investigation.

## AI can draft and sort in investigations

| Task | AI role | Person's role |
| --- | --- | --- |
| Draft a timeline | Builds it from reports and notes, with a source for each entry | Checks it against records and interviews |
| Suggest questions and evidence | Lists interview questions and records to pull | Decides what to ask |
| Find clusters | Groups events by location, shift, equipment and cause | Judges if the pattern is real |
| Draft contributing factors | Writes first drafts in cause, effect, event form | Team rewrites to the rules. See the [root cause analysis and CAPA guide](https://incidentkit.ai/guides/root-cause-analysis-and-capa-guide) |
| Summarize long records | Produces a short draft | Checks what it left out |
| Propose actions | Lists options ranked by strength | Owner and leaders choose. RCA2 asks top leaders to approve each action |

## AI should not decide, close or contact

- **Set final severity or harm class.** It drives escalation and outside reporting.
- **Decide if a deadline applies.** A model can remind. A person decides.
- **State a root cause or assign fault.** Causes are findings, not predictions.
- **Recommend discipline.** That belongs in HR, outside root cause analysis.
- **Close or verify a corrective action.** Verification needs real-world evidence.
- **Contact patients, families or regulators.** A named person must do it.
- **Change the record silently.** Every AI edit must be visible and reversible.

See [incident reporting in healthcare](https://incidentkit.ai/guides/incident-reporting-in-healthcare) for deadlines.

## Hallucination is a real risk

NIST's Generative AI Profile calls false output confabulation: confidently stated but erroneous content. It follows from how models work: they predict likely text. NIST adds that people tend to over-trust automated output, which it calls automation bias.

A 2025 npj Digital Medicine study tested clinical note generation across 18 configurations and 12,999 clinician-annotated sentences. It found a 1.47 percent hallucination rate and a 3.45 percent omission rate. Refining prompts and workflows brought major errors below previously reported human note-taking rates.

That study covers clinical notes, not incident intake, so it is not a benchmark for any product. Three lessons carry over. Error rates are not zero. Omissions can outnumber inventions. Design and testing change the result.

A missing fact, such as a witnessed fall, can matter as much as an invented one.

| Failure | Example | Control |
| --- | --- | --- |
| Fabrication | A draft gives a time the reporter never stated | Fill only from the reporter's words; ask when something is missing |
| Omission | A summary drops that a bed alarm was off | Show the original beside the draft; reviewer opens the source |
| Misattribution | An action goes to the wrong person or shift | Confirm names and roles |
| Overconfident cause | Draft says the cause was failure to follow policy | Treat causes as drafts; require team approval |
| Transcription error | A wrong dose in a voice transcript | Reporter confirms names, doses and times |

## Test AI on your own incidents

Do not rely on a vendor demo or figures from other tasks. Test on your own text, and set the pass mark before you see results.

1. **Collect real cases** Pull 50 to 100 past narratives, including a few hard ones. Use a BAA or proper de-identification.
2. **Set the pass mark first** Decide what rate of inventions, omissions and wrong categories you accept, and which errors are never acceptable.
3. **Compare drafts with signed records** Have clinicians mark each difference: invention, omission, misattribution or harmless rewording.
4. **Test the review step too** Do reviewers catch planted errors? A rubber-stamp review makes any draft unsafe.
5. **Repeat after every change** Re-run when the vendor changes the model, prompts or form.

## Human-in-the-loop design that holds up

1. **Preserve the reporter's account** Store what was written or dictated, unchanged.
2. **Label AI output as draft** AI-filled fields stay marked and unapproved until a person acts.
3. **Review each field** A named reviewer approves, edits or rejects each one.
4. **Sign by a named person** A person signs the report or investigation.
5. **Log the whole chain** The audit trail records the AI suggestion, the human edit, who approved it and when.
6. **Audit samples** Compare drafts with final records on a schedule to measure edit rates and error types.

The last step counters automation bias. Put the most friction where a wrong draft hurts most: severity, cause and reportability.

IncidentKit follows this design. [Lauren](https://incidentkit.ai/product/lauren) asks follow-up questions, fills the form and drafts the investigation. Every AI-drafted field shows "Lauren · draft" until approved, and a person always reviews, edits and signs. Staff report by text now. Voice and human-authored RCA templates are rolling out.

## HIPAA and BAA: the model provider counts too

If narratives contain patient information, the vendor is a business associate. HIPAA names patient safety activities (42 CFR 3.20) as covered functions. The agreement must include the terms in 45 CFR 164.504(e). See [business associate agreement](https://incidentkit.ai/glossary/business-associate-agreement) and [HIPAA](https://incidentkit.ai/hipaa).

The AI model provider is the next link. A subcontractor that handles protected health information for the vendor is itself a business associate, and the vendor's contract must bind it to the same limits.

Ask which provider processes your data, whether a BAA covers that exact service, and whether your data trains models. Return-or-destroy terms cover logs and backups.

- Add the AI feature to your HIPAA risk analysis.
- Send the model only what the task needs.
- Ask for a breach notice window far shorter than 60 days.
- With a patient safety organization, ask counsel about your evaluation system.

## Log enough to rebuild who decided what

HIPAA requires mechanisms that record and examine activity in systems holding electronic protected health information, and controls against improper alteration. For AI features, keep enough to reconstruct who decided what.

| Event | Record |
| --- | --- |
| AI draft generated | Source text, model and version, output as shown, time |
| Field edited | Who, when, old and new value |
| Field approved | Who, when, which draft |
| Record or investigation signed | Signer, time, full state at signing |
| AI feature changed or disabled | Who, when, setting before and after |

Store each draft as it was shown, because models change. Retention runs for years: OSHA records five years, process safety reports five years, HIPAA policy documents six years. See the [audit trail](https://incidentkit.ai/product/audit-trail).

## Accreditors now write AI expectations

In September 2025 the Joint Commission and the Coalition for Health AI released guidance with seven elements: AI policies and governance, patient privacy and transparency, data security, ongoing quality monitoring, voluntary blinded reporting of AI safety events, risk and bias assessment, and training.

The fifth element points back to your incident system. AI errors are reportable events. Make sure staff can file one, and review them like any other.

AAAHC's v45 standards (August 2026) add an AI governance framework: leadership accountability, risk assessment, cybersecurity, human oversight and transparency. They apply to surveys on or after December 15, 2026. AAAHC's guide includes an AI inventory and vendor checklist.

Add your incident software's AI to the inventory. See [AAAHC](https://incidentkit.ai/compliance/accreditation/aaahc) and [Joint Commission](https://incidentkit.ai/compliance/accreditation/joint-commission).

## Questions to ask vendors

### Design

- Which fields can the AI draft, and which can it never change?
- Is every AI-drafted field marked until a person approves it?
- Can a reviewer see the original text beside each draft?
- Can we turn each AI feature off?

### Data

- Which model providers process our data, and where?
- Does a BAA cover that exact service?
- Is our data used to train models, by you or your provider?
- What do you retain, and how is data deleted at exit?

### Evidence

- What do you log for each AI action?
- How do you measure fabrication and omission on incident text?
- What happens to drafts and tests when the model changes?

Pair these with the questions in the [buyer's guide](https://incidentkit.ai/guides/incident-management-software-buyers-guide).

## Frequently asked questions

### Can AI write an incident report?

AI can draft one from the reporter's account: ask follow-up questions, fill fields and structure the narrative. A person must review, edit and sign it. Keep the reporter's words unchanged, mark AI-filled fields as drafts until approved, and record who approved what.

### Is it safe to put patient information into an AI tool?

Only if the vendor has signed a business associate agreement and the AI model provider that processes the data is also bound by one. Ask whether your data trains models, what is kept, and where it is processed. Without a BAA covering the exact service, do not enter protected health information.

### What is AI hallucination, and how is it controlled?

Hallucination, which NIST calls confabulation, is confidently stated but false output. Controls: fill fields only from the reporter's words, ask instead of guessing, link each field to its source, show the original beside drafts, require human approval, and sample drafts against final records.

### Will AI replace risk managers or investigators?

No. Judging severity, finding causes, disclosing to families, deciding what to report and verifying fixes all need accountable people. Joint Commission and AAAHC guidance stresses governance and human oversight. AI changes the first draft, not who is responsible.

### Should AI decide severity or whether something is reportable?

No. It can suggest, and it can remind a reviewer that a deadline may apply. A named person should set severity and decide on reporting. Rules vary by setting and state, and a wrong call can mean a missed clock or an unneeded report.

### How do we audit AI-assisted incident reports?

Log every draft, edit and approval with who and when. Sample records regularly. Compare AI drafts with the final signed record, and track edit rates and error types such as omissions and invented details. Feed findings back into settings, training and vendor review.

## Sources

- [NIST, Artificial Intelligence Risk Management Framework: Generative AI Profile (NIST AI 600-1)](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf)
- [npj Digital Medicine, A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation (2025)](https://www.nature.com/articles/s41746-025-01670-7)
- [Joint Commission Perspectives, November 2025 (Joint Commission and CHAI responsible AI guidance)](https://digitalassets.jointcommission.org/api/public/content/ac1009e032dc4632a317c35e5ddc86fe)
- [AAAHC, v45 Standards press release (August 18, 2026)](https://www.aaahc.org/uploads/2026/08/260817_MBD_DOC_AAAHC-v45-Standards-Press-Release_FINAL.pdf)
- [AAAHC, 1095 Advance AI Governance Guide press release (August 17, 2026)](https://www.aaahc.org/uploads/2026/08/2608014_MBD_DOC_1095-Advance-AI-Governance-Guide_Press-Release_FINAL.pdf)
- [45 CFR 160.103, Definitions, including business associate (eCFR)](https://www.ecfr.gov/current/title-45/section-160.103)
- [45 CFR 164.504, Business associate contracts (eCFR)](https://www.ecfr.gov/current/title-45/section-164.504)
- [45 CFR 164.410, Notification by a business associate (eCFR)](https://www.ecfr.gov/current/title-45/section-164.410)
- [45 CFR 164.308, Administrative safeguards, including risk analysis (eCFR)](https://www.ecfr.gov/current/title-45/section-164.308)
- [45 CFR 164.312, Technical safeguards (eCFR)](https://www.ecfr.gov/current/title-45/section-164.312)
- [45 CFR 164.316, Policies, procedures and documentation requirements (eCFR)](https://www.ecfr.gov/current/title-45/section-164.316)
- [42 CFR 3.20, Patient safety work product definitions (eCFR)](https://www.ecfr.gov/current/title-42/part-3)
- [IHI and NPSF, RCA2: Improving Root Cause Analyses and Actions to Prevent Harm](https://www.ihi.org/sites/default/files/RCA2_ImprovingRootCauseAnalysesandActionstoPreventHarm.pdf)

## Related

- [Lauren: AI incident intake that a person signs](https://incidentkit.ai/product/lauren)
- [How AI intake changes incident reporting: forms vs chat](https://incidentkit.ai/blog/how-ai-intake-changes-incident-reporting)
- [Incident audit trail: every change, who and when](https://incidentkit.ai/product/audit-trail)
- [IncidentKit security overview](https://incidentkit.ai/security)
- [HIPAA and BAA for incident reporting](https://incidentkit.ai/hipaa)
- [Business associate agreement: definition and meaning](https://incidentkit.ai/glossary/business-associate-agreement)
- [Incident management software buyer's guide: how to choose](https://incidentkit.ai/guides/incident-management-software-buyers-guide)
- [Incident reporting in healthcare: the complete guide](https://incidentkit.ai/guides/incident-reporting-in-healthcare)