Databricks — Teaching a Search Agent When to Stop
Episode notes
Single-step retrieval is fast but weak on multi-hop questions, and step-by-step search is accurate but slow. Databricks trained a search agent to decide when it has searched enough: frontier-level quality in 5.8 seconds.
Key takeaways:
- The trade-off. Instructed-Retriever-1 handled most requests with fast single-step search, but multi-hop questions need iterative searching that adds latency.
- Learn when to stop. With online RL (CISPO, Clipped Importance Sampling Policy Optimization), the agent is rewarded for good results and penalized for extra steps that don't pay off.
- Built-in guardrails. A fixed upper bound caps sequential steps, and synthetic multi-hop questions teach it when more searching helps.
- Pick your operating point. Tuning the step penalty traces a Pareto frontier from faster to more thorough.
- Results. It matches leading models' quality while answering in 5.8 seconds on average, more than 2x faster than Claude Sonnet 5, DeepSeek-V4-Flash and GPT-5.6 Luna. These are vendor-reported benchmarks.
Chapters:
00:00 The hidden wait in enterprise AI
01:32 More compute isn't the fix
03:04 Why single-step fails multi-hop
04:36 The cost of sequential search
05:22 Online RL and CISPO
06:55 Synthetic multi-hop questions
07:42 Deciding when to stop
08:27 A hard cap on steps
09:13 The Pareto frontier
10:47 5.8 seconds
12:17 How novel is it?
13:50 Fan-out vs. adaptive search
17:39 Missing details
19:12 Recap
Sources:
- Adaptive Instructed-Retriever: Frontier-Quality Search at 2x Lower Latency (Databricks Blog, Sep 2026)
Related episode: E24: Google — Making AI Search Ask Better Questions, 20x Faster
Enjoyed it? Follow Data Science in the Wild (one real production ML/AI system or paper per episode, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.
Also from Peter: The GenAI Evolution Atlas, Wednesday AI deep dives and Friday news roundups.
Made with AI-generated voices; curated by Peter Liu from the sources above.