NOTES / 2026.03.21 · 4-post thread · 45 likes
Generation Scales. Verification Doesn't.
Verifying whether a paper is a breakthrough requires roughly the same expertise as producing one. AI science is the extreme case: infinite generation, fixed verification bandwidth.
The answer is you can’t — not cheaply. Verifying whether a paper is a breakthrough requires roughly the same expertise as producing one. Generation scales. Verification doesn’t. x.com
We call this the Measurability Gap. The harder a task is to verify relative to how hard it is to execute, the less safely you can automate it. AI science is the extreme case: infinite generation, fixed verification bandwidth.
We formalize this, and map the conditions under which verification becomes the binding constraint on AI progress, in Section 5: arxiv.org
“Reality is the ultimate eval” is exactly right. The question is whether world models can internalize enough physics to run that eval without a human expert in the loop. If they can’t, we have a ceiling. x.com
Originally published as a thread on X.