Presentation
Evaluating Negotiation Capabilities of Large Language Models: From Ultimatum Games to Nash Bargaining
SessionPoster Session 2
DescriptionNegotiation is a live, back-and-forth process—exactly the kind of human interaction today’s static AI benchmarks miss. We created interactive agent environments based on two classic game-theory paradigms—the one-shot Ultimatum Game and the open-ended Nash Bargaining task—to watch large language models (LLMs) reason, cooperate, and compete as the deal keeps changing. Using the Harvard Negotiation Project’s six principles (Interests, Legitimacy, Relationship, Options, Commitment, Communication) we scored a variety of large language models across hundreds of rounds. Llama-3 generally struck the most effective bargains; Claude-3 leaned aggressive—maximizing its own gain but risking push-back—while GPT-4 offered the fairest splits. The results spotlight both promise and pitfalls: today’s top LLMs can already secure mutually beneficial deals, yet still falter on consistency, legitimacy, and commitment when stakes rise. Our open-source benchmark invites human-factors researchers to probe these behaviours, design safer negotiation workflows, and study how mixed human-AI teams might unlock even better outcomes.
Event Type
Poster
TimeWednesday, October 15th5:30pm - 6:30pm CDT
LocationRiverside East
Similar Presentations