← Learning paths

10 lessons · take it step by step

AI Reliability & Hallucinations

Diagnose retrieval and generation failures, verify evidence and measure the tradeoff between answers and abstention.

Start here

Follow the order if you are new, or jump straight to a topic you need.

Start the first lesson →

  1. Hallucinations & Failure Taxonomy

    25–40 min

    A wrong answer may arise from missing knowledge, a retrieval miss, malformed output, a failed tool or an unsupported generated claim.

  2. Reliability Evaluation & Metrics

    25–40 min

    Answer coverage is answered cases divided by relevant cases; false refusal is abstention on cases annotated as answerable.

  3. Grounding & Claim-Level Verification

    25–40 min

    Split a response into checkable claims and bind each to evidence.

  4. Retrieval Failure Analysis

    25–40 min

    Separate a knowledge gap from a ranking miss.

  5. Abstention & Uncertainty

    25–40 min

    Use ABSTAIN when authorized evidence is insufficient, ask a clarifying question when ambiguity can be resolved, and route to a human when the task exceeds the product scope.

  6. Verification Tools & Structured Answers

    25–40 min

    Use arithmetic for numbers, compilation or pinned API documentation for code, and authorized reference data for facts.

  7. Conflicting & Stale Evidence

    25–40 min

    Two documents can disagree because they describe different versions or effective dates.

  8. LLM Judge Calibration & Bias

    25–40 min

    An automated judge can be sensitive to answer order, verbosity, style and injected evaluator instructions.

  9. Long Context & Multilingual Reliability

    25–40 min

    Long inputs can truncate relevant evidence or distract the generator with unrelated content.

  10. Reliability User Experience

    25–40 min

    A visible citation link can help inspection, but a polished badge can also encourage unjustified trust.