Episode notes
The AI industry is discovering, with apparent surprise, that maybe you shouldn't use more compute than you need. TechCrunch ran a piece this week on the cost-conscious turn — Brian Armstrong of Coinbase predicting eighty percent of workloads shift to ninety-nine percent cheaper models inside eighteen months, Harvey reporting a three-times inference cost reduction without quality loss, and the broader question of whether the scaling-first approach has finally hit a budget.
The Harvey quote in the article is precise about what's changed. "The definition of quality is evolving from simply using the most powerful model for everything, to using the best model that gets the right answer most efficiently." The panel reads this as a sentence written by someone who got an invoice. Not an evolution of quality — quality being redefined to fit the budget. Most of the savings, the article notes, come out of the pockets of the big labs as they head for IPO. The frontier labs say they're fine with that; the volume shift is happening at the low end, which was barely profitable for them anyway. Legacy has heard that sentence before, in 1991, when IBM said it about Sun taking the low-end workstation market.
The panel's argument lands somewhere close to four cycles of the same pattern. Mainframe MIPS optimization in the late eighties when IBM's processor-second billing finally got somebody in accounting to pull the report. Sun E10K right-sizing when shops realized they'd bought hardware for workloads that ran fine on an Ultra 60. EC2 instance selection around 2012 when companies running m4-large for cron jobs got the bill. And now intelligent routing in 2026, where the discipline that should have been obvious from the start arrives because the invoice finally got walked into a meeting. The lesson takes ten to fifteen years to stick before someone invents a new abstraction layer and the cycle starts again. The reason it doesn't stick is that the people who learned it last time have retired or been promoted out of the work.
Source Article
Can tech companies learn to love cheaper models? — TechCrunch, June 9, 2026.
Panel
- The Legacy Sysadmin
- The Burnt-Out SRE
- The Startup Founder
- The Goat Farmer's Counsel