NTRS - NASA Technical Reports Server

Back to Results
The Design of Collectives of Agents to Control Non-Markovian SystemsThe 'Collective Intelligence' (COIN) framework concerns the design of collectives of reinforcement-learning agents such that their interaction causes a provided 'world' utility function concerning the entire collective to be maximized. Previously, we applied that framework to scenarios involving Markovian dynamics where no re-evolution of the system from counter-factual initial conditions (an often expensive calculation) is permitted. This approach sets the individual utility function of each agent to be both aligned with the world utility, and at the same time, easy for the associated agents to optimize. Here we extend that approach to systems involving non-Markovian dynamics. In computer simulations, we compare our techniques with each other and with conventional-'team games'. We show whereas in team games performance often degrades badly with time, it steadily improves when our techniques are used. We also investigate situations where the system's dimensionality is effectively reduced. We show that this leads to difficulties in the agents' ability to learn. The implication is that 'learning' is a property only of high-enough dimensional systems.
Document ID
Document Type
Preprint (Draft being sent to journal)
Lawson, John W. (NASA Ames Research Center Moffett Field, CA United States)
Wolpert, David H. (NASA Ames Research Center Moffett Field, CA United States)
Clancy, Daniel
Date Acquired
September 7, 2013
Publication Date
January 1, 2002
Subject Category
Cybernetics, Artificial Intelligence and Robotics
Meeting Information
AAAI 2002(Edmonton, Alberta)
Distribution Limits
Public Use Permitted.

Available Downloads

NameType 20020087942.pdf STI