Skip to content

Evaluation report · Hallucination audit

Path Audit: The Missing work/ Directory ​

Conditional·Repository audit

TL;DR ​

The Phase 2 brief said the content loader should read work/*.md, cases/*.md and evals/*.md. Two of those paths existed after scaffolding; work/ never did. A model that "helpfully" creates the directory would fork the source of truth. This audit checked the filesystem first, mapped Work to the existing projects/ collection, and kept every published URL intact.

Claim under test ​

The loader can read work/*.md.

Method ​

  1. List tracked files and directories under the project root.
  2. Check whether work/, cases/ and evals/ exist.
  3. Compare the declared globs with the actual paths.
  4. Inspect how Work URLs are published today.
  5. Choose the smallest change that satisfies the intent without creating a second copy of the content.
bash
find . -maxdepth 2 -type d \
  -not -path './node_modules*' \
  -not -path './.git*' \
  -not -path './.vitepress/dist*'

Finding ​

Declared pathExists?Actual pathDecision
work/*.mdNoprojects/*.mdMap to kind: 'work'
cases/*.mdYes (created)cases/*.mdUse as-is
evals/*.mdYes (created)evals/*.mdUse as-is

There was no hidden work/ directory, no symlink and no previous rename in git history. The Work label in the navigation refers to /projects/, which already serves the eight published project pages in both languages.

Decision ​

The loader owns the mapping:

ts
const SOURCES = [
  'projects/*.md', // exposed as kind: 'work'
  'cases/*.md',
  'evals/*.md',
  // zh/ mirrors...
]

This keeps:

  • existing /projects/... URLs and inbound links stable;
  • one copy of every project page;
  • a single card schema across Work, Cases and Evals.

It does not satisfy the literal work/*.md path from the brief.

Verdict ​

Conditional. The intent — one loader spanning work, cases and evals — is met, and no hallucinated directory was created. The literal path requirement is not met.

A future rename is possible, but it must be a deliberate migration: create work/, move the files, add redirects from /projects/*, update the sidebar and nav, then delete the old directory. Doing that silently inside a loader would break links and split the source of truth.

Evidence ​

CheckResult
Declared collections3
Existing collections2
Mapped collections1
Invented directories0
Broken Work links0
Duplicate project pages0

Crystaria (with Paw and Kyle)