Eval harness

The module

Lesson, exercise and rubric

• Read the lesson
• Do the exercise
• Check it against the rubric
• Lab 01-rag-pipeline
• Lab 04-eval-harness

Open the repository in a coding agent and run /module 04 for a Socratic session over this stage.

What you leave behind · code

Eval harness

Required sections:

• fixtures
• assertions
• baseline run
• failure exit code

May appear on a public portfolio once redacted: remove production fixture data.

The eval · eval:harness-fails-on-regression

The harness can actually fail

1. nonzero-exitA deliberately broken fixture makes the harness exit non-zero. Checked by: Run the harness against the seeded regression fixture; require exit code != 0.
2. fixtures-versioned-apartFixtures are not edited in the same commit as the system under test. Checked by: Diff the submitted commit; fixtures and prompts must not both change.

Every check must pass. There is no partial credit and no override.

Evidence that counts

Proof has to exist outside your own claim.

At least 2 of: a commit or file a third party can open; a recorded, reproducible eval run. Evidence older than 365 days is stale and does not count.