When an LLM gets a fact wrong, is the fact missing (an "empty shelf") or present-but-unretrievable ("lost keys")? Jordan and Riley unpack Google Research's argument that standard accuracy metrics collapse these two cases — even though one calls for scaling model/data and the other for cheaper post-training/inference-time fixes. They walk through "knowledge profiling" (analyze the state of each fact, not each question) and its five profiles, the three behavioral notions (encoding via priming, knowledge across phrasings incl. the reversal curse, and recall — direct vs. inference-without-encoding), and WikiProfile, a 2,150-fact benchmark (10 tasks each) built with a Gemini-2.5-Pro + search-grounded pipeline. Headline finding on frontier models (Gemini 3, GPT-5): many factual errors are recall failures, not encoding failures — the knowledge is often  ...