Evidence

The verified-program benchmark

Does an agent writing Bosatsu and checking it with our tools reach a program that passes a hidden harness on fewer tokens than one writing TypeScript from scratch? Four forum features, seventeen properties, and a harness neither arm sees decide.

The recorded result (iteration 001, 2026-09-05): the gate fails on every pair

Three fresh sonnet agents per arm per task, twenty-four runs. The Bosatsu arm established 16 of 17 properties on every run; the TypeScript arm established 16, 17, and 17. The Bosatsu arm spent roughly twice the tokens: tokenLift (TypeScript tokens per Bosatsu token) 0.54, 0.50, and 0.44 on the three run pairs. The pre-registered gate is accuracy(A) ≥ accuracy(B) and lift above 1 on each pair; it fails on all three, and the number is published as it came.

pairverdictaccuracy, Bosatsu + toolsaccuracy, TypeScripttokens, Bosatsutokens, TypeScripttokenLift
run 1fail0.9410.941527,348286,8490.54
run 2fail0.9411.000482,343242,7530.50
run 3fail0.9411.000540,819238,4180.44

Both misses are on the concurrent-reply task; the record attributes them to incomplete evidence, not an observed failing run.

Ask your agent about anything here; it has the detail.

Where this fits

What this is about: Why an Analyzable Language?