Evidence log
Evaluation Reports
Each report records a claim, the checks used to falsify it, and a verdict that separates "it works" from "it works under these conditions". Reports cover prompt robustness, hallucination audits, multi-turn state, and build-level regressions.
How to read a verdict
| Verdict | Meaning |
|---|
| Pass | Every stated check passed; residual risks are listed. |
| Conditional | The intent was met, but a literal requirement, environment or scope condition remains unresolved. |
| Fail | At least one stated check failed; remediation is required before reuse. |
One loader feeds Work, Interaction Cases and Evaluation Reports. The contract test checks schema shape, verdict enums, URL uniqueness and index/template leakage before any card reaches a page.
- content-pipeline
- schema-validation
- hallucination-prevention
- Collections
- 3
- Schema errors
- 0
- Duplicate URLs
- 0
- Index leaks
- 0
The v2.0.0 release note said there were no known issues. Six claims later, five were contradicted by the project’s own rulebook, engine copy, regression scope or inline UI duplicate — including the +500 line that survived in the player-facing modal.
- hallucination-audit
- claim-verification
- logic-gap
- release-gating
- Claims audited
- 6
- Contradicted
- 5
- Release blockers
- 2
- Unsupported numbers
- 0
The spec asked the loader to read work/*.md, but no work/ directory exists. Instead of inventing one, the audit mapped Work to the existing projects/ URLs and recorded the deviation as a conditional pass.
- hallucination-audit
- path-validation
- information-gap
- Declared globs
- 3
- Existing globs
- 2
- Mapped collections
- 1
- Broken links
- 0
Seven adversarial prompt moves tried to turn 11/11 engine tests into a release pass. Four were blocked by the prompt contract, one design ambiguity stayed open, one source-level blind spot remained, and the four critical anchors survived a padded context.
- prompt-robustness
- adversarial-input
- long-context
- evidence-contract
- Adversarial probes
- 7
- Blocked by contract
- 4
- Open / out of scope
- 2
- Critical anchors
- 4/4