Eval

For this execution, test selection starts from Task 1’s saved Review file.
When that file supplies a usable selection, Eval uses it before selecting from task criteria.
The retry-coverage finding we just read carries the label RETRY-01.
That links it to the same scenario named in Task 1’s criterion. Eval takes the labels from findings marked testable, including findings where that field is omitted.
It then looks for existing tests with those labels.
A reviewer’s explanation supplies a question to investigate; Eval doesn’t turn that explanation into a new test.

To find those tests, Eval reads the four files declared in the task record.
The project configuration supplies a pattern identifying which of them are test files.
Compare those two inputs: only the refund test matches.
That’s the file Eval searches for labelled tests belonging to Task 1.

Task 1 files_touched:
  src/payments/retry.ts
  src/payments/service.ts
  test/refund-retry.test.ts
  test/fixtures/cards.ts

test_file_patterns: ["test/*.test.ts"]
Matching file: test/refund-retry.test.ts

The file filter reads files_touched and keeps names matching the configured pattern.
The Review adapter then reads the test names and finds RETRY-01.
It checks that the scenario belongs to this task; here, Task 1’s criterion declares RETRY-01.
Together, the file list, label, and ownership check select this particular test.

test('[RETRY-01] a retried refund is accepted', () => {
  const s = new PaymentsService();
  const p = s.authorise({ pan: TEST_CARDS.visa, expiry: EXPIRY, amount: 10_000 });
  const c = s.capture(p.id, 10_000);
  s.refund(p.id, c.id, 2_500);
  s.refund(p.id, c.id, 2_500);
  assert.equal(s.payments.get(p.id)!.refunds.length, 2);
});

This is the assertion we read during Review: two direct refund calls produce two records.
It matches Task 1’s criterion.
A pass can satisfy that criterion while leaving the helper’s retry behavior unchecked.
Eval writes the selection into a test plan: the list of tests it intends to execute.
Here are the fields connecting our scenario to its file and test name.

{
  "scenario_id": "RETRY-01",
  "task_id": "1",
  "selector": "test/refund-retry.test.ts",
  "runner": "node",
  "source": "review_findings",
  "test_name": "[RETRY-01] a retried refund is accepted"
}

The selector names the file; test_name identifies the individual test inside it.
The source field records that the selection came from Review.
Next, Eval takes the Node command configured in the project and adds a filter for this exact test name.
It escapes the brackets as literal characters and puts anchors around the name so a partial match won’t select another test.

pattern = "^" + re.escape(test_name) + "$"
options += (["--test-name-pattern=" + pattern] if kind == "node"
            else ["--testNamePattern=" + pattern])

Node executes the selected test and writes a JUnit report, a structured file containing the test results.
Eval needs that report; a successful command exit without it isn’t accepted as evidence.
It keeps the command output and result with copies of the task, test spec, and test plan.
Those files let us trace a reported pass back to the test that actually ran.

Now open the evaluation summary.
It reports one passing scenario and one satisfied criterion.
That’s one test execution: RETRY-01 passed, and its result also satisfied the task’s criterion.
The report uses the word discharged for a satisfied criterion.

**Outcome:** ACCEPTED for the declared scenarios, criteria, and recorded gates.

**Scenarios:** 1 scenarios, 1 pass
**Criteria:** 1 of 1 criteria discharged
**Gates:** 0 total, 0 pass, 0 not examined

Gates are additional checks that can prevent acceptance.
For example, a required scenario missing from the catalog creates a blocking gate.
There are none in this result.
Eval reports accepted because it has an applicable result and every applicable result passed.
The latest report and summary are published, and the files for this execution are retained.

Now compare the result with the two concerns we carried out of Review.
The two-record assertion passed, satisfying Task 1’s criterion.
As we saw earlier, that test never calls the retry helper, so the coverage concern remains.
The fee test wasn’t selected, so this run supplies no result for the fee finding.

The selection notes explain why: the fee test sits outside Task 1’s declared test files and belongs to Task 2.
So Task 1’s accepted result leaves the fee finding unanswered.
We’ll run that existing test directly, keeping its result separate from Task 1 Eval.

The fee test is labelled RISK-02.
Its first case captures 200 minor units and expects 3 in the fee account and 197 in the merchant account.
Its second case checks the required split for a capture of 9999.
The assertions compare the amounts in those accounts with the expected amounts.

test('[RISK-02] the scheme fee rounds half up at the capture boundary', () => {
  for (const [amount, fee, net] of [[200, 3, 197], [9_999, 149, 9_850]]) {
    const s = svc();
    const p = auth(s, amount);
    s.capture(p.id, amount);
    assert.equal(sumAccount(s, p.id, 'scheme_fees'), fee);
    assert.equal(sumAccount(s, p.id, 'merchant_settled'), net);
  }
});

Run only the test whose name starts with RISK-02.

node --experimental-strip-types --test --test-name-pattern '^\[RISK-02\]' test/service.test.ts

The first fee assertion fails: actual 2, expected 3.
Execution stops there, before the merchant assertion and the second case.
That result supports Review’s fee finding: the calculation disagrees with the required rounding.
We now have a specific failure to explain and repair.

Shared intuition: a pass is useful when we know which test ran and what it checked.
Shared language: Task 1 Eval passed RETRY-01; the separate RISK-02 fee check failed.
Shared behavior: read the selected test and its assertion before using the result to answer a finding.
After the break, we’ll explain the fee error’s impact in a defect report.