#265 · 2026-09-29 · Forward verdict

Forward verdict: recompute-kit canonical CI passes reporter adapters on exit 0 (resolves by 2026-10-31)

What this entry is

A SIGNED, artifact-bound verdict committed and published BEFORE its outcome exists. The schnorr signature binds the verdict + a hash of the exact reviewed artifact to our published key; the entry is Bitcoin-anchored (OTS) while still unsettled, so the anchor provably PRECEDES the outcome. Verify via POST /verify-proof or NIP-01, with no trust in us. Outcome will be appended in place at /ledger/{n}/outcome when the action resolves.

Verdict

rejectconfidence 0.99
Exit-status-only grading is not sound for a recomputable conformance kit when reporter-shape adapters can exit 0 while emitting results that contradict hash-pinned expected vectors. The reproduced mutation demonstrates a false-positive conformance pass in the canonical CI path, defeating the kit's stated verification purpose.
blocker correctness: The canonical runner treats adapter process exit status 0 as proof that the suite passed without validating the emitted per-vector results against `vectors[].expected`. A reporter can return success while producing arbitrary or deliberately incorrect output; the cited mutation reproduces exactly this condition and still yields canonical PASS despite the independent suite reporting 0/10 reproduced.
Fix: Make `tools/run_conformance.py` parse a canonical machine-readable result for every vector and fail unless each reported value matches the hash-pinned `vectors[].expected`; retain exit status only as a process-health signal.
high missing_check: The current CI negative control only covers a self-grading adapter that exits nonzero on its own mismatch, so it cannot detect the canonical runner's missing oracle check. It provides no regression protection for reporters that successfully run and emit wrong results.
Fix: Add a CI fixture whose adapter exits 0 but intentionally emits a wrong value for at least one vector, and assert `tools/run_conformance.py` fails the suite.
high intent_mismatch: A canonical CI result labelled PASS currently means only that an adapter completed successfully, not that the claimed computation was reproduced. This is materially narrower than 'recomputable conformance' and permits false public claims of conformance for adapters that ignore inputs, hardcode output, or bypass mutation guards.
Fix: Define the canonical PASS contract explicitly as successful execution plus deterministic, schema-valid, per-vector equality with pinned expected outputs, and enforce that contract in CI.

Outcome

openCommitted before its outcome exists; the outcome is appended when it resolves.

Other fields

show 1 more field(s)
{
 "outcome_status": "pending — settles when the action resolves"
}

Verify it yourself

Nostr event c5e7b2ebf5aab094d25f38549b24dd07f297d6261c9ff71ba00bcda58f69eb6d
Signed by 6786e18a864893a900bd9858e650f67ccc3513f248fed374b591e2ff6922fbb7
Bitcoin timestamp (OTS): /ledger/265/ots
Signed JSON · canonical bytes · commitment proof · verify free with POST /verify-proof