Presentation
A New Evaluation Metric for Takeover Maneuver Quality: Comparing Human Drivers with Autonomous Driving Agents
DescriptionAssessing the quality of driving maneuvers, especially in safety-critical scenarios such as the transition from automation to human control (takeover) in conditionally automated vehicles, remains a critical challenge for enhancing road safety. Existing metrics for evaluating takeover performance suffer from limitations: many rely on discrete or subjective assessments that may inadequately capture subtle differences in driving maneuvers over time. Most current performance metrics are “snapshots” of the takeover process, such as response time, lateral and longitudinal speed, acceleration jerk, and time-to-collision (TTC). This emphasizes the need for more robust performance measures. To address this gap, the present study introduces the Trajectory Integral Index (TII), a novel metric that compares human driving trajectories, including both speed and position changes, with context-specific references generated by a state-of-the-art autonomous driving algorithm (Transfuser++). A driving simulator (CARLA) study was conducted, involving seven participants using a 2 (5- vs. 10-second takeover lead time) × 2 (visual-cognitive distraction) within-subjects design. Results from an Aligned Rank Transform ANOVA revealed that the TII effectively captured significant main effects of distraction (p = .0040), urgency (p = .0158), and their interaction (p = .0268). Specifically, distracted drivers in urgent scenarios exhibited greater trajectory deviation compared to the AI reference. In contrast, conventional metrics failed to detect these differences. Despite the modest observed effect size (statistical power of 65%), the findings demonstrate the feasibility of using AI-agent-generated reference maneuvers as a sensitive and valuable benchmark for evaluating driving maneuver quality, particularly during takeovers under varying of urgency and distraction.
Authors
Assistant Professor
Event Type
Late Breaking Results
TimeWednesday, October 15th3:40pm - 4pm CDT
LocationGrand Hall G/H
General Sessions
Similar Presentations