Data Science in the Wild
Temporada 1
Meta — One Model Replaces the Whole Retrieval Stack
Google — Ending the Six-Hour Forecast Gap
Instacart — The Death of the Static Shopping Page
Shopify: An AI Agent That Trains Itself
por Peter Liu
A deep dive into SilverTorch, the system Meta built to collapse recommendation retrieval — normally split across separate CPU services for approximate nearest-neighbor search, feature filtering, and scoring — into a single GPU-native PyTorch model under a new "Index as Model" paradigm. We cover the business problem (network overhead and duplicated infrastructure from stitching together microservices as retrieval architectures grew more complex), the technical approach (a model-based GPU Bloom filter for candidate filtering fused with an Int8-quantized approximate-nearest-neighbor kernel, plus unified scoring layers that replace what used to be separate services), and the reported production payoff: a 23.7x throughput improvement and 13.35x cost-efficiency gain over the prior CPU-based approach, now running across hundreds of retrieval models serving billions of daily active users. Source article: "SilverTorch: Index as Model — A New Retrieval Paradigm for Recommendation Systems" — Engineering at Meta, https://engineering.fb.com/2026/05/26/ml-applications/silvertorch-index-as-model-new-retrieval-paradigm-recommendation-systems/ (published 2026-05).
A deep dive into WeatherNext 3, Google DeepMind and Google Research's third-generation global weather AI model. We cover the business context (why a forecast that only updates every six hours is a real operational cost for renewable energy, agriculture, aviation, and emergency response), the technical approach (a single Functional Generative Network mesh-transformer that ingests live geostationary satellite imagery to refresh its forecast every hour instead of every six, at roughly five times the resolution of its predecessor), and what the reported accuracy gains — up to 60% better precipitation forecasts against satellite benchmarks — actually mean in practice, plus where the write-up leaves real questions open. Source article: "Introducing WeatherNext 3, Our Most Advanced and Accurate Global Weather AI Model" — Google (DeepMind / Google Research), https://blog.google/innovation-and-ai/models-and-research/google-deepmind/introducing-weathernext-3/ (published 2026-09).
A deep dive into how Instacart rebuilt its Shopping Hub — the discovery page shoppers see right after picking a retailer — around a four-phase generative pipeline instead of a static, manually curated content library. We cover the business problem (personalization that couldn't scale, and a page built by siloed teams that never felt cohesive), the technical approach (an LLM agent that designs page themes top-down, retrieval-augmented keyword generation grounded in the real catalog, layered LLM-judge and embedding-based quality filters, and a cheap distilled classifier that cuts evaluation cost by an estimated 99%), and what the early results show — and don't yet show — about putting a generative model in charge of page structure itself. Source article: "Our Early Journey to Transform Instacart's Discovery Recommendations with LLMs" — Instacart Tech, https://tech.instacart.com/our-early-journey-to-transform-instacarts-discovery-recommendations-with-llms-cf4591a8602b (published 2026-02).
A deep dive into how Shopify rebuilt the improvement loop for Sidekick, its AI agent, as a four-stage flywheel that keeps getting better without perpetually scaling frontier-model spend. We cover the business context (quality that gets expensive fast when you only buy it by upgrading models), and the technique: a human-calibrated quality rubric (validated with Cohen's kappa), an LLM-as-judge tuned with DSPy/GEPA/ACE and — crucially — validated against real production A/B outcomes before it's trusted, automated prompt/tool research, and continual learning via SFT plus GRPO fine-tuning that uses the calibrated judge itself as the reward signal. The payoff: serving cost for a GraphQL agent handling 2,000 requests/min fell roughly 96%, from ~$27M to ~$1M/year. The real lesson is the discipline of calibrating the judge before using it as a reward. Source article: "Sidekick's Continual Learning Loop" — Shopify Engineering, https://shopify.engineering/sidekicks-continual-learning-loop (published 2026-08).
A deep dive into how LinkedIn replaced its DCNv2 feed ranker with Feed SR — a decoder-only transformer that reads up to 1,000 of a member's past impressions as an ordered sequence, served on a disaggregated CPU/GPU stack with custom kernels. We cover the business context (a 1.2B-member feed whose legacy ranker couldn't model history as a sequence), the technical approach (RoPE positional embeddings, a parallel-DCNv2 scoring head, shared-context batching, and a custom flash-attention CUDA kernel), the honest negative results (why a fine-tuned LLM-ranker and a TransAct-style encoder were tried and rejected), and the realized impact: +2.10% time spent and +3.52% social actions in a production A/B test — at roughly 0.7× the inference energy of the CPU-served incumbent despite being a larger model. Source article: "An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking (Feed SR)" — LinkedIn / arXiv, https://arxiv.org/abs/2602.12354 (published 2026-02).