Presentation
Building an LLM-Based Teammate in Minecraft: A Testbed for Human-AI Collaboration
SessionPoster Session 2
DescriptionCommercial-Off-The-Shelf (COTS) games and Large Language Models (LLMs) are enabling new empirical paradigms in the study of human-machine teams (HMTs). COTS games that allow for modifications are lowering barriers to the design and conduct of controlled experimental testbeds, while advances in LLMs have dramatically broadened the scope of possible interaction modes between humans and machines. In this presentation, we describe our novel testbed design process in developing a Minecraft tower defense game to investigate the impacts of agent and team composition on HMT performance and team processes. Our study builds on insights from the DARPA Artificial Social Intelligence for Successful Teams (ASIST) in designing two versions of LLM-enabled teammates that work alongside humans in a task-performer role. We discuss lessons learned and challenges encountered in implementing interactive LLM agents within a real-time, action-oriented game scenario, addressing game design trade-offs between difficulty and team performance measure validity, and ongoing experimentation efforts using remote data collection protocols.
Contributors
Event Type
Poster
TimeWednesday, October 15th5:30pm - 6:30pm CDT
LocationRiverside East