Agent S family of computer-use agents (CUAs)
AI Cutting Edge by Sujith
Episode notes
These sources chronicle the evolution of the Agent S family of computer-use agents (CUAs), detailing improvements in performance and methodology across several iterations. Agent S introduced hierarchical planning, memory, and a constrained action space to tackle complex desktop tasks, achieving a state-of-the-art (SoTA) result of 20.6% on the OSWorld benchmark. Agent S2 significantly advanced this by implementing Proactive Hierarchical Planning and a Mixture-of-Grounding strategy, pushing the SoTA to 48.8%. Finally, Agent S3 incorporated a native coding agent and a wide-scaling framework called Behavior Best-of-N (bBoN), which generates multiple rollouts and uses concise "behavior narratives" for principled traje ...
Keywords
Artificial IntelligenceComputation and LanguageComputer Vision and Pattern Recognition Machine Learningmemoryplanningcua computer-automationmllmretrieval-augmented-generationin-context-reinforcement-learning, agent-computer-interfacegui-agents computer-useomputer-use-agent Behavior Best-of-N (bBoN) OSWorldWindowsAgentArenaAndroidWorldScaling Agentsbehavior narrativestrajectory selectionAgent SAgent S2Agent S3 Mixture-of-GroundingMoGProactive Hierarchical Planningflat policynative coding agent foundation modelsLLMsadvanced reasoningautonomous agentsAGIGPTAIOpsfederated learning trustworthy AIUI designGUIUICaliforniaCanadaFloridaNew YorkTexasuser agentsUnited KingdomWashington user agentsinnovative navigationBill GatesSam Altmansandbox environments Ilya SutskeverIEEE MIT Technology Review cyber attacksdata integrityOpenAI's ChatGPTDALL-EKai-Fu Leeincident management stargate datasetsdeepfake prevention open-sourceagentic frameworksLinkedIn
Where this episode is made
Country