Edward Hu on Inventing LoRA, MuTransfer & The Compute Revolution

(2:05) LoRA origins — Edward describes the practical need that led to the creation of LoRA at Microsoft in 2021 while collaborating with OpenAI

(6:50) Why LoRA worked — Low-rank updates to large models made fine-tuning efficient and widely accessible

(9:45) Biological analogy — LoRA as the equivalent of learning a new instrument with shared prior knowledge

(11:35) Industry adoption — Edward reflects on companies thanking him for saving compute costs and resource usage

(13:30) Intelligence as compression — Discussion of whether brains and AI share a compression-based structure for learning

(17:15) Risks of over-biological inspiration — Why not everything in AI  ...