NASA Logo

NTRS

NTRS - NASA Technical Reports Server

Due to the lapse in federal government funding, NASA is not updating this website. We sincerely regret this inconvenience.

Back to Results
Dynamic Programming for Structured Continuous Markov Decision ProblemsWe describe an approach for exploiting structure in Markov Decision Processes with continuous state variables. At each step of the dynamic programming, the state space is dynamically partitioned into regions where the value function is the same throughout the region. We first describe the algorithm for piecewise constant representations. We then extend it to piecewise linear representations, using techniques from POMDPs to represent and reason about linear surfaces efficiently. We show that for complex, structured problems, our approach exploits the natural structure so that optimal solutions can be computed efficiently.
Document ID
20040086785
Acquisition Source
Ames Research Center
Document Type
Preprint (Draft being sent to journal)
Authors
Dearden, Richard
(NASA Ames Research Center Moffett Field, CA, United States)
Meuleau, Nicholas
(NASA Ames Research Center Moffett Field, CA, United States)
Washington, Richard
(NASA Ames Research Center Moffett Field, CA, United States)
Feng, Zhengzhu
(Massachusetts Univ. Amherst, MA, United States)
Date Acquired
September 7, 2013
Publication Date
January 1, 2004
Subject Category
Systems Analysis And Operations Research
Meeting Information
Meeting: Twentieth Conference on Uncertainty in Artificial Intelligene (UAI-04)
Location: Banff
Country: Canada
Start Date: July 7, 2004
End Date: July 11, 2004
Funding Number(s)
CONTRACT_GRANT: NCC2-1311
CONTRACT_GRANT: NSF IIS-02-19606
Distribution Limits
Public
Copyright
Work of the US Gov. Public Use Permitted.
No Preview Available