This technical document, the Agents Companion, explores the advancements in generative AI agents, highlighting their architecture composed of models, tools, and an orchestration layer, moving beyond traditional language models. It emphasizes Agent Ops as crucial for operationalizing these agents, drawing parallels with DevOps and MLOps while addressing agent-specific needs like tool management. The paper thoroughly examines agent evaluation methodologies, covering capability assessment, trajectory analysis, final response evaluation, and the importance of human-in-the-loop feedback alongside automated metrics. Furthermore, it discusses the benefits and challenges of multi-agent systems, outlining various design patterns and their application, particularly within  ...