Measure results

Measure results

Measure the three course goals separately. Completion, satisfaction and confidence can help improve the course, but they do not demonstrate that the course achieved its goals.

1. Measure shared intuition

Give every participant the same unfamiliar scenario before instruction. Ask each person to identify the risks, the evidence they would seek and the action they would take.

Repeat the exercise with a comparable scenario at the end of the course.

Look for:

• Greater agreement about the important risks
• Greater agreement about where human review is required
• Clear separation of facts, claims and unknowns
• Decisions supported by evidence instead of confidence

Do not require identical answers. The goal is a compatible mental model and a consistent standard of evidence.

2. Measure shared language

Ask participants to classify examples with the course vocabulary. Include cases that require them to distinguish:

• A requirement from an acceptance criterion
• A test from validation
• Evidence from acceptance
• Context from a verified fact
• A recommendation from a decision
• A review comment from a grounded finding

Score both the selected term and the explanation. A correct label with an incorrect rationale does not demonstrate understanding.

3. Measure behavior change

Observe whether participants use the workflow without being reminded. During the final exercise, look for the following actions:

• Identify missing context before implementation
• Convert important constraints into testable criteria
• Stop when a consequential requirement is unclear
• Write or select a test that can fail for the expected reason
• Review AI-generated work against the specification
• Record the evidence and the decision separately
• Assign each finding a clear disposition

Measure the same behaviors again in normal work after the course.

Evidence plan

When	Method	What it measures
Before the course	Independent scenario response	Initial intuition and vocabulary
During each exercise	Predictions, artifacts and explanations	Understanding and guided performance
End of course	Unfamiliar transfer exercise	Independent performance
Two weeks later	Short scenario and repository review	Early retention and use at work
30 to 60 days later	Sample of specifications, tests and reviews	Sustained behavior change

Minimum success criteria

The course meets its goals when:

1. Most participants identify the same high-consequence risks in the final scenario.
2. Participants use the core terms correctly and support classifications with evidence.
3. Participants complete the transfer exercise without a prescribed prompt.
4. Work samples collected after the course show continued use of the target workflow.

Set numeric thresholds after a pilot establishes a credible baseline. Do not choose a percentage only because it appears precise.

Course improvement data

Collect participant feedback about pace, clarity, accessibility and technical setup. Keep this data separate from learning and behavior measures. A participant can enjoy the course without learning the material, and a demanding course can produce strong learning.

Use the results to revise activities, instructions and facilitation. Do not lower a behavior standard to improve satisfaction scores.

Exercise traceability check

Review each exercise before it enters the course and after each pilot.

Check	Pass condition
Goal alignment	The exercise advances at least one of the three course goals.
Process alignment	The exercise supports Define, Plan, Execute, Judge or Learn.
Participant action	The participant makes a decision or produces work.
Evidence	The exercise creates something that can be reviewed.
Feedback	The participant receives timely information about the result.
Reinforcement	The course asks the participant to use the skill again.

An exercise fails the traceability check if either its goal or Workbench stage is missing.