Evidence
Benchmark methods, results, and raw artifacts
The examples show the tools working; these benchmarks ask whether they help.
Read all of it as a self-run benchmark: the same project builds the tools, generates the challenges, and scores the runs, most cells are single runs, and failed rounds stay in the record.
1. Fact-tooling review from computed facts and source
Do AI reviewers find more planted defects with computed program facts than by reading source? Eleven rounds over codebases from 2,028 to 46,922 lines are recorded. In round 008, the largest, the fact-using reviewer found 25 of 26 and the source reader 23 of 26; the record keeps the rounds the facts lost.
The fact-tooling scoreboard defines its columns and renders every recorded round. The served round 008 JSON contains the headline source and fact scores.
2. Program-report and source arms on the same quiz
Can an agent answer review questions from a program’s report instead of its source?
On a 30-question quiz about nine programs both arms scored 29/30. The report arm
spent 87,074 tokens against the source arm’s 114,291. The measured
tokenLift is 1.31. It is one recorded probe.
gate.json records the
accuracy comparison, token lift, and gate verdict.
artifact-score.json
records the report arm, and
baseline-score.json
records the source arm.
3. Reflection-auth review with and without a counterexample
Does the checker’s counterexample change what a weaker model certifies? From source alone a haiku-tier model called a flawed authentication server SECURE; with the counterexample it said VULNERABLE. Three cells with one trial each show the case exists, not a rate.
iteration-009.md contains the
three prompts, model responses, labels, and token counts. It is
committed under benchmarks/dist/results/.
The distributed-systems benchmark page
describes task 010, the other task shapes, and the execution semantics.
4. A verified program in Bosatsu with the tools, or in TypeScript from scratch
Does writing Bosatsu with the tools reach a verified program on fewer tokens than TypeScript?
No: the gate fails on all three pairs. The Bosatsu arm established 16 of 17 properties on every
run against the TypeScript arm’s 16, 17, and 17, and spent roughly twice the tokens
(tokenLift 0.54, 0.50, 0.44).
The verified-program benchmark page carries
the per-pair table, where the tokens went, and the two misses;
summary.json is the
served artifact every figure above is read from.
5. UI update throughput against React
Does compile-time knowledge of state-to-DOM dependencies reduce runtime work? The harness runs in your browser and measures how quickly each runtime accepts updates, not frame rate; no cross-machine speedup is claimed.
Run the comparison and inspect its linked Bosatsu sources; follow the compiler’s binding map through an actual counter update.
Ask your agent about anything here; it has the detail.