What is AI hallucination risk?

How hallucination occurs and why it persists

Language models can generate plausible statements that are unsupported or false. Search, retrieval and tools can provide better evidence, but the system can still select the wrong source or misinterpret it. An answer may mix accurate and inaccurate material, and a citation is useful only if it supports the claim. Confidence of wording is not a reliability measure. For consequential work, check the source and any calculation independently and decide what level of human review is needed.

Human verification protocols for managing hallucination risk

Managing hallucination risk in a business context requires three elements. First, a clear policy on what AI outputs can be used for without verification and what requires verification before use. AI outputs used for internal brainstorming or drafting carry different risk than AI outputs used in client communications or regulatory submissions. The verification requirement should be proportionate to the consequence of an error. Second, a qualified human reviewer: verification is only meaningful if the reviewer has the expertise to identify errors. An AI-generated legal summary cannot be verified by someone without legal knowledge. The verification protocol must identify who is competent to verify each type of AI output. Third, an audit trail: documenting that verification occurred; who checked, when, what was checked; creates accountability and provides evidence of due diligence in the event that an error is later identified. The EU AI Act’s human oversight requirement for high-risk AI reflects these principles: genuine human oversight, not nominal review.

Legal references reviewed 4 October 2026. High-risk provisions have phased application and transition rules. Sources: Commission timetable, deployer duties, impact-assessment scope and literacy guidance.