NASA Logo

NTRS

NTRS - NASA Technical Reports Server

Back to Results
Agent Reward Shaping for Alleviating Traffic CongestionTraffic congestion problems provide a unique environment to study how multi-agent systems promote desired system level behavior. What is particularly interesting in this class of problems is that no individual action is intrinsically "bad" for the system but that combinations of actions among agents lead to undesirable outcomes, As a consequence, agents need to learn how to coordinate their actions with those of other agents, rather than learn a particular set of "good" actions. This problem is ubiquitous in various traffic problems, including selecting departure times for commuters, routes for airlines, and paths for data routers. In this paper we present a multi-agent approach to two traffic problems, where far each driver, an agent selects the most suitable action using reinforcement learning. The agent rewards are based on concepts from collectives and aim to provide the agents with rewards that are both easy to learn and that if learned, lead to good system level behavior. In the first problem, we study how agents learn the best departure times of drivers in a daily commuting environment and how following those departure times alleviates congestion. In the second problem, we study how agents learn to select desirable routes to improve traffic flow and minimize delays for. all drivers.. In both sets of experiments,. agents using collective-based rewards produced near optimal performance (93-96% of optimal) whereas agents using system rewards (63-68%) barely outperformed random action selection (62-64%) and agents using local rewards (48-72%) performed worse than random in some instances.
Document ID
20060022168
Acquisition Source
Ames Research Center
Document Type
Conference Paper
Authors
Tumer, Kagan
(NASA Ames Research Center Moffett Field, CA, United States)
Agogino, Adrian
(California Univ. Santa Cruz, CA, United States)
Date Acquired
August 23, 2013
Publication Date
January 1, 2006
Subject Category
Systems Analysis And Operations Research
Meeting Information
Meeting: AAMAS''06
Location: Hokkaido
Country: Japan
Start Date: May 8, 2006
End Date: May 12, 2006
Distribution Limits
Public
Copyright
Other

Available Downloads

There are no available downloads for this record.
No Preview Available