NOTES / 2026.03.22 · 9-post thread · 55 likes
Karpathy's Throwaway Caveat Is the Hard Ceiling
Karpathy described the limit on trillion-dollar autonomous systems as an aside about an overnight hyperparameter script: objective metrics are the perfect fit — and everything else isn't.
FIG. 01 — FROM THE ORIGINAL THREAD
Karpathy just described the hard ceiling on trillion-dollar autonomous systems as a throwaway caveat about his overnight hyperparameter script (@NoPriorsPod) — and went back to tuning his learning rate…
@karpathy: “This is extremely well suited to anything that has objective metrics that are easy to evaluate… writing kernels for more efficient CUDA code […] are the perfect fit. Perfect fit.”
“But many things will not be. […] If you can’t evaluate then you can’t auto research it.”
Nobody in the conversation reacts to this. He’s describing a property of his hyperparameter tuner the way you’d describe a property of your dishwasher. Works great for plates, not for cast iron… x.com
Except the “cast iron” is every autonomous system whose output a human can’t cheaply verify. So, you know. Most of the economy.
What he actually said is: the bottleneck on autonomous AI is not intelligence. It’s not compute. It’s not data. It’s whether anyone can check the answer. That’s it. That’s the whole economics of AGI!
Every lab is racing to remove humans from the loop. The entire economy depends on keeping them in it. No one has reconciled this. We tried.
The cost to automate is collapsing. The cost to verify is bounded by biology. That single asymmetry drives everything: alignment drift, the collapse of the junior pipeline, liability-as-software, the future of work, and where value will ultimately accrue.
Paper: arxiv.org
Originally published as a thread on X.