Presentation
VirTLab-Eval: Human-AI Team and Digital Twin Performance Evaluation Demonstration
SessionDemonstrations 2
DescriptionThe study of human-AI teaming (HAT) and Human Digital Twin (HDT) modeling currently faces significant challenges in accurately simulating and assessing the effectiveness of interactions between humans and AI systems. Current methods typically rely on limited real-world data or simplified simulated representations that do not capture the complexity and variability of human digital twin and AI agent behavior. VirTLab-Eval is a novel agentic framework of human digital twins (HDTs) and AI agents to model and evaluate HAT behaviors and interventions across operational scenarios. A comprehensive set of measures, include novel socio-cognitive-emotional and behavioral metrics automatically extracted from team communications and interactions, for evaluating HAT performance is integrated into innovative visual analytics enabling the assessment of interventions that target the attributes of HDTs and our AI agents. VirTLab-Eval measurements capture nuanced aspects of trust development, emotional alignment, and team cohesion that conventional performance metrics might miss, providing richer insights into the HAT dynamics. Example results on search and rescue missions reveal that AI teammate reliability significantly impacts communication dynamics and team assistance behaviors, while HDT personality traits influence trust development and team coordination - insights that directly inform the design of HAT training programs.
Event Type
Demonstration
Industry/Practitioner Content
TimeWednesday, October 15th6pm - 6:30pm CDT
LocationRiverside East
Similar Presentations