As large language models (LLMs) continue to improve in accuracy and capability, they are increasingly deployed in user-facing conversational applications. Despite these advances, LLM-based systems remain susceptible to unpredictable behaviors, including hallucinations, unsafe outputs, and inconsistent or contextually inappropriate responses. In conversational multi-agent systems, such failures can propagate rapidly across interactions and agents, amplifying risk for both users and enterprises. Ensuring trustworthiness in these systems therefore requires not only more accurate models, but also effective strategies for real-time monitoring, evaluation, and mitigation to maintain reliable and safe operation.
Most current research and industrial practice focuses on offline evaluation, post-deployment monitoring, and periodic human review. While valuable, these approaches are insufficient for interactive systems operating under strict latency constraints, where delayed intervention can already result in user harm or degraded customer experience. A key open challenge is real-time mitigation in conversational multi-agent systems, where autonomous AI agents interact and coordinate with one another. Addressing this challenge requires architectures in which AI agents collaborate with human agents to generate, evaluate, and, when necessary, correct responses on the fly, ensuring both safety and high-quality user interactions.
This talk focuses on real-time human–AI collaborative architectures for building trust and managing risk in conversational generative AI systems. Drawing on our company’s decades of experience deploying large-scale, real-time conversational AI systems with human-in-the-loop for millions of customers in enterprise customer care environments, we explore how established principles of human–machine collaboration can be adapted and extended to generative and multi-agent systems, and where new design paradigms are needed to enable seamless interaction.
We describe system architectures in which AI and human agents bring specialized, complementary expertise to produce high-quality conversational responses. AI agents may include customer-facing agents responsible for communication and guidance, transactional agents handling bookings or payments, orchestration agents coordinating multi-agent workflows, and evaluation agents monitoring outputs for confidence, policy compliance, or risk. Similarly, human agents contribute diverse skills, including domain knowledge, familiarity with enterprise policies, and the ability to review and correct unacceptable or high-risk model outputs. This diversity enables nuanced, context-aware responses: AI agents provide real-time assistance through summarization, intent detection, emotion and sentiment analysis, and customer experience signals, while human agents can operate behind the scenes to review or approve AI-generated outputs without disrupting conversation flow. When automated mitigation is insufficient, human agents can seamlessly take over to de-escalate issues or handle high-risk scenarios. Beyond real-time collaboration, closed-loop feedback mechanisms leverage multiple end-to-end measures—including AI performance, efficiency, and customer experience—to continuously optimize the system. Reinforcement learning and adaptive escalation strategies allow both AI behavior and human workflows to improve over time, creating a dynamic, self-improving human–AI collaborative ecosystem.
The talk concludes with lessons learned from real-world deployments and data-driven insights, highlighting key trade-offs among accuracy, latency, cost, and user experience, as well as open research and standardization challenges at the intersection of signal processing, human-centered AI, and industrial-scale multi-agent conversational systems. The discussion emphasizes actionable guidance for designing real-time human–AI collaborative systems that integrate multiple AI and human expertise, effectively manage risk, and continuously improve through feedback. The goal is to provide researchers and practitioners with practical insights from real-world applications to build robust, scalable conversational AI systems capable of operating safely and efficiently in complex, dynamic environments.
