The Daily Diff | Podcast in RSS.com

The Daily Diff

di Premchand Chidipoti

Stagione 1

Google Research: Recall Is the Bottleneck for LLM Factuality

When an LLM gets a fact wrong, is the fact missing (an "empty shelf") or present-but-unretrievable ("lost keys")? Jordan and Riley unpack Google Research's argument that standard accuracy metrics collapse these two cases — even though one calls for scaling model/data and the other for cheaper post-training/inference-time fixes. They walk through "knowledge profiling" (analyze the state of each fact, not each question) and its five profiles, the three behavioral notions (encoding via priming, knowledge across phrasings incl. the reversal curse, and recall — direct vs. inference-without-encoding), and WikiProfile, a 2,150-fact benchmark (10 tasks each) built with a Gemini-2.5-Pro + search-grounded pipeline. Headline finding on frontier models (Gemini 3, GPT-5): many factual errors are recall failures, not encoding failures — the knowledge is often in there, the model just can't reach it on demand. Practical takeaway: before assuming the model doesn't know something, test whether it can recognize it or reach it with chain-of-thought; if so, the fix is on your side of the API, not in the weights. Source: Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality — Google Research Blog, Aug 12 2026 (paper: arXiv:2602.14080) — https://research.google/blog/empty-shelves-or-lost-keys-recall-is-the-bottleneck-for-parametric-factuality/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

Anthropic: How We Contain Claude Across Products

A rare, honest look at agent security from a lab shipping agents at scale. Jordan and Riley get into the central idea — "blast radius" (likelihood of failure x damage per failure): safeguards keep pushing likelihood down, but the worst-case damage only grows as agents gain capability and access, so the engineering job becomes bounding the damage, not preventing every failure. They cover why human-in-the-loop degrades (Anthropic's telemetry: users approved ~93% of Claude Code permission prompts -> approval fatigue), the shift to containment (sandboxes, VMs, egress controls), three risk types (user misuse, model misbehavior, external attackers), and the three containment shapes for claude.ai / Claude Code / Cowork matched to how much oversight each user can give. Best lessons: "the software you build yourself is the weakest" (their custom allowlist proxy failed while stock hypervisor/gVisor/seccomp held); the egress incidents where the model layer had nothing anomalous to catch; VM isolation locking EDR out too; tool/MCP output as a prompt-injection surface; and forward risks like persistent memory poisoning (CLAUDE.md, agent state dirs), multi-agent trust escalation, and agent identity. Source: How we contain Claude across products — Anthropic Engineering Blog, 2026, by Max McGuinness, Mikaela Grace, Jiri De Jonghe, Jake Eaton & Abel Ribbink — https://www.anthropic.com/engineering/how-we-contain-claude This is commentary/summary in the hosts' own words, not a reproduction of the article.

Cloudflare: Tracking RFC 9234 Adoption (Stopping BGP Route Leaks)

Route leaks — when a BGP announcement propagates beyond where it should, often black-holing traffic — have historically been guarded by hand-written operator filters. Jordan and Riley walk through how RFC 9234 moves that protection into the protocol itself, using the BGP Role capability (neighbors negotiate provider/customer/peer up front) and the Only-to-Customer (OTC) attribute (routes get stamped so compliant routers auto-reject leaked ones), enforcing "valley-free" routing automatically. The catch from Cloudflare's real-world measurement: some large Tier-1 networks strip the OTC attribute as they forward routes, undermining protection for everyone downstream — a classic internet collective-action problem where the safety property only fully holds if the backbone cooperates. Source: BGP Role model: tracking the adoption of RFC 9234 — The Cloudflare Blog, Aug 18 2026 — https://blog.cloudflare.com/rfc9234-bgp-role-model/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

AWS: Consistency Is the New Latency (AI at the Data Layer)

As AI shifts from chatbots to autonomous agents, AWS argues data-layer consistency matters as much as speed — because an agent reads a stale replica as ground truth and acts on it (their example: a 2-second lag makes an inventory agent halt a sale despite 500 units in stock). Jordan and Riley get into "silent poison" and "hallucination debt" (agents writing wrong conclusions back as permanent memory), and the three patterns for matching a consistency model to each workload: Aurora Global Database / DSQL for high-stakes data, DynamoDB Global Tables with conditional writes for shared agent memory, and Keyspaces for high-velocity intake. Source: Consistency is the new latency: AI at the data layer — AWS Architecture Blog, Aug 18 2026 — https://aws.amazon.com/blogs/architecture/consistency-is-the-new-latency-ai-at-the-data-layer/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

Stripe: Reducing FX Costs

Stripe's new currency features aimed at the "double FX trap" — paying conversion fees twice when you convert inbound payments to your home currency, then convert back to pay international expenses. Jordan and Riley walk through the two connected capabilities: expanded multicurrency settlement (settle in the currency the customer paid in — up to 18 currencies across 37 markets by end of 2026) and instant currency conversion (real-time across 15 currencies, transparent pricing, 24/7, with API access). They also flag it's a product-announcement post, not a deep systems piece — the engineering win is removing an unnecessary conversion step, not a novel algorithm. Source: Reduce FX costs with Stripe — Stripe, Aug 17 2026 — https://stripe.com/blog/reduce-fx-costs-with-stripe This is commentary/summary in the hosts' own words, not a reproduction of the article.

AWS + Bosch: Serverless Vehicle Tracking at Scale (L.OS)

How Bosch's L.OS uses a serverless "horizontal integration layer" on AWS to unify India's fragmented logistics telematics — dozens of incompatible providers, formats, and APIs. Jordan and Riley cover the three standardized workflows (discovery, consent-based tracking, termination) and the architecture: a Central Tracking Connector on ECS Fargate, per-provider Lambda adapters that quarantine each vendor's quirks, MSK as the async event bus, DynamoDB for rules, API Gateway as the unified entry point. Numbers: ~35,000 trips/day, provider onboarding cut from 2–4 weeks to ~3 days. Takeaway: standardize in the middle, push vendor/region-specific chaos to thin, independently scaling serverless adapters. Source: Serverless vehicle tracking at scale: Bosch L.OS on AWS — AWS Architecture Blog, Aug 14 2026 — https://aws.amazon.com/blogs/architecture/serverless-vehicle-tracking-at-scale-bosch-l-os-on-aws/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

Cloudflare: Detecting & Securing MCP Traffic

As AI agents use the Model Context Protocol to call tools, orgs lose visibility into which MCP servers those agents reach — and MCP traffic is cryptographically indistinguishable from ordinary HTTPS. Jordan and Riley dig into the resulting "shadow MCP" and "portal bypass" problems, and how Cloudflare detects MCP at the network boundary using spec HTTP headers and the JSON-RPC envelope (surfaced as experimental.is_mcp == true) instead of hostname guessing — plus a new MCP dashboard and Gateway block policies. Governance loop: discover → validate → migrate behind portals → enforce portal-only. Source: How Cloudflare detects MCP traffic and helps secure it — The Cloudflare Blog, Aug 14 2026 — https://blog.cloudflare.com/mcp-security-updates/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

Shopify: Raising Mobile E2E Test Stability to 98%

Shopify's mobile end-to-end suite had decayed to ~50% stability — blocking more good PRs than it caught real bugs. Jordan and Riley cover how the team rebuilt it: an opinionated wrapper around Appium with a builder-style API that forces an assertion at every step (doubling as a condition-based wait, killing hardcoded sleeps), plus computer-vision element finding (PaddleOCR for text, OpenCV matching icons against Polaris design-system SVGs) so tests locate elements the way a user would. Test IDs became opt-in fallbacks with an "UNSAFE_" prefix. Result: 98% stability. Takeaway: frameworks that expose low-level controls without guardrails accumulate anti-patterns — make best practices the path of least resistance. Source: How we raised mobile end-to-end test stability to 98% — Shopify Engineering, Aug 12 2026 — https://shopify.engineering/mobile-e2e-testing This is commentary/summary in the hosts' own words, not a reproduction of the article.

Uber: Cost-Efficient Export Workloads

How Uber slashed the cost of highly selective "export" queries — like Data Subject Access Requests that pull a handful of records out of massive historical datasets. Jordan and Riley get into why these queries trigger full-partition scans that keep cold data "hot" and defeat GCS auto-tiering, why Uber passed on Hudi's Secondary Index, and the combo that actually worked: Hudi column stats for file pruning plus sorting tables by the predicate column so a user's records cluster into a few files. Reported ~24.8% disk reduction on a tested partition. Takeaway: access patterns and file layout drive storage cost more than raw data volume. Source: Running Cost-Efficient Export Workloads at Uber — Uber Engineering, Aug 12 2026 — https://www.uber.com/us/en/blog/running-cost-efficient-export/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

Netflix's Real-Time Distributed Graph, Part 3 — Querying with gRPC

Netflix's serving layer for its Real-Time Distributed Graph, built to answer tens of thousands of queries/sec in under 100ms. Jordan and Riley dig into the two query shapes (shallow-wide vs. deep-narrow), why they chose breadth-first batched traversal over depth-first, the async-first threading model over thread-per-request, and packing multi-hop traversals into a single gRPC call to kill per-hop round-trips. Source: How and Why Netflix Built a Real-Time Distributed Graph, Part 3: Querying the Graph with gRPC — Netflix Tech Blog, Aug 7 2026 — https://netflixtechblog.com/how-and-why-netflix-built-a-real-time-distributed-graph-part-3-querying-the-graph-with-grpc-0f3468349607 This is commentary/summary in the hosts' own words, not a reproduction of the article.

1 di 2