Work through a repeatable loop: diagnose the diff, choose review methods, evaluate evidence gaps, define defect specs, plan, run and repeat the first three stages on the new diff.
Current fixture behavior still exposes diagnose, plan, validate and report. V1 targets diagnose → review → eval → define → plan → run. Stage 2 selects methods before results are shown, and Stage 7 remains blocked until the unseen assessment change exists.
Learning outcomes
The stages succeed when practitioners recognize the same risks, describe them in the same words and change what they do before approving work.
Outcome 01
Engineers recognize common AI failure modes and understand why different risks require different validation methods.
ObservableGiven an AI-generated change, learners identify plausible failure modes before selecting a method.
Outcome 02
Teams consistently describe failure modes, validation methods, evidence strength, unexamined areas and human decisions.
ObservableLearners explain a validation decision using the same terms for risk, method, evidence and uncertainty.
Outcome 03
Before approving AI-generated code, engineers diagnose risk, judge evidence, define bounded repairs and repeat the loop after every new diff.
ObservableLearners perform the workflow on an unfamiliar change without a prescribed tool list.
The learning path
Each stage adds one decision and produces an artifact the next stage works from. Preparation happens before instructional time.
Sibling repository: ../payments-validation-fixture. Complete this before the first stage.
Run workbench diagnose, inspect the AI-authored refund-retry diff and decide which failure modes are plausible before selecting a review or security method.
Choose security and code-review methods before seeing their results, then run workbench review to test whether the diagnosed risks are supported by the diff and requirements.
Run workbench eval on the review output, find gaps in the tests and evidence, and separate code defects, test gaps and specification gaps.
Run workbench define to turn newly identified code defects and test gaps into bounded, auditable defects/*.md files. Route policy gaps to an accountable human instead of inventing requirements.
Run workbench plan to turn defects/*.md into tasks.json: an executable implementation and validation plan with explicit checks, gates, budget and omissions.
Run workbench run with tasks.json, apply only the approved repair scope and inspect the new implementation diff and run output.
After run creates a new implementation diff, repeat workbench diagnose, workbench review and workbench eval independently before approving, defining another defect or escalating policy.
Failure classes and the checks that examine them
Current versus V1
The target sequence is explicit about what exists now, what the new stages require and where delivery is still blocked.
The current fixture exposes diagnose, plan, validate and report. validate executes the workflow and report summarizes its evidence. Results may appear before Stage 2 method selection.
diagnose → review → eval → define → plan → run. Review methods are chosen before results are shown; define writes defects/*.md; plan writes tasks.json; run creates the next diff. Stage 7 is blocked until the unseen assessment change exists.
Where this sits
Tracks at the same number are peers on the same fixture. The 501 assumes both of them.