AI Hallucinations & Confabulations

1. Concept Overview & Systemic Problem

The most dangerous property of artificial intelligence lies in the gap between linguistic perfection and factual accuracy:

1. Illusion of Competence: The model writes about a fabricated API method with the same confident, academic tone as it does about the official documentation of a standard library.
2. Dependency Hallucination: In an attempt to solve a developer's problem, the LLM often imports a convenient method or library that never existed in the real world.
3. Hallucination Cascades: If an autonomous agent makes its first decision based on a hallucination, all subsequent 10 steps will attempt to develop this non-existent branch of reality, wasting hours of time and budget.

Hallucinations are not a bug of a specific neural network but an inseparable trait of autoregressive probabilistic systems, requiring engineers to implement a strict system of external grounding and deterministic validation.

2. Architectural Taxonomy & Mental Model

In artificial intelligence, hallucinations are classified into three main forms of manifestation:

• 1. Intrinsic Hallucinations: The model's response directly contradicts information already provided in the prompt or RAG context (e.g., the text states a price of $50, while the model concludes with $150).
• 2. Extrinsic Hallucinations: The model adds details that were not present in the context and do not exist in reality (fabricated legal articles, links to non-existent pages, quotes from people).
• 3. Code & API Hallucinations:
  • Fabricated CLI command flags (e.g., docker run --auto-clean).
  • Non-existent class methods (e.g., calling db.users.findAndUpsert()).
  • Fabricated third-party npm/pip packages (Slopsquatting threat).

3. Technical Pipeline & Internal Mechanics

The engineering pipeline for suppressing and detecting hallucinations includes:

1. Context Grounding: Utilizing strict RAG with a prompt constraint: “Respond exclusively based on the provided documents. If the answer is not in the text, return 'NO_DATA'.”
2. Sampling Parameter Tuning: Setting temperature = 0.0 and lowering top_p to minimize random wandering in the tails of the probability distribution.
3. Deterministic Pre-flight Verification: Automatic oracle invocation: compiling TypeScript (tsc), checking for the presence of imported packages in the official registry via API or HTTP HEAD requests to URL links.
4. Adversarial Double-Check: A lightweight judge model compares the output text with source documents, marking any unverified statements (Fact Extraction & NLI Matching).

4. Production Engineering Scenarios

01. Pre-flight Audit of Dependency Registries (Protection Against Slopsquatting)

An agent writes a script and adds a new package to package.json. Before executing npm install, the security system queries registry.npmjs.org. If the package was registered less than 48 hours ago or does not exist, the action is immediately blocked with an alert for suspected hallucination or attack.

02. RAGAS Grounding in Corporate Support Assistant

A chatbot responds to employees' legal inquiries. The framework automatically checks each generated assertion. If the Faithfulness level drops below 0.90, the user sees a message: “I cannot find the exact clause in the regulations, please contact the HR department.”

03. Schema Verification with OpenAPI / Swagger

Before calling an external API, the code agent is required to match the function name and its arguments with the swagger.json schema specification file, eliminating fabricated query parameters.

5. Pitfalls, Common Mistakes & Security

• Blind Trust in Beautiful Style: The more proficient the model is in language, the easier it is for a human to believe its fabrications. Always demand precise references to sources or execution artifacts.
• Hallucinations of Regulatory and Medical Norms: Attempting to obtain advice on tax codes without connecting to the current text of the law via RAG will inevitably lead to legal penalties.
• Artificial Hallucinations Due to Context Rot: Outdated or noise-filled context increases the frequency of hallucinations by 3–5 times. Keep the context clean.