Jefferson Provost, Benjamin J Kuipers, and Risto Miikkulainen. Self-Organizing Perceptual and Temporal Abstraction for Robot Reinforcement Learning. In AAAI-04 Workshop on Learning and Planning in Markov Processes, 2004.
A major current challenge in reinforcement learningresearch is to extend methods that work well on discrete, short-range,low-dimensional problems to continuous, highdiameter, high-dimensionalproblems, such as robot navigation using high-resolution sensors. Wepresent a method whereby an robot in a continuous world can, withlittle prior knowledge of its sensorimotor system, environment, andtask, improve task learning by first using a self-organizing featuremap to develop a set of higher-level perceptual features whileexploring using primitive, local actions. Then using those features,the agent can build a set of high-level actions that carry it betweenperceptually distinctive states in the environment. This methodcombines a perceptual abstraction of the agent's sensory inputinto useful perceptual features, and a temporal abstraction of theagent's motor output into extended, high-level actions, thusreducing both the dimensionality and the diameter of the task. Anexperiment on a simulated robot navigation task shows that the agentusing this method can learn to perform a task requiring 300small-scale, local actions using as few as 7 temporally-extended,abstract actions, significantly improving learning time.
@InProceedings{Provost-aaai04-ws9,
author = {Jefferson Provost and Benjamin J Kuipers and Risto Miikkulainen},
title = {Self-Organizing Perceptual and Temporal Abstraction for Robot Reinforcement Learning},
booktitle = {AAAI-04 Workshop on Learning and Planning in Markov Processes},
year = 2004,
abstract = { A major current challenge in reinforcement learning
research is to extend methods that work well on discrete, short-range,
low-dimensional problems to continuous, highdiameter, high-dimensional
problems, such as robot navigation using high-resolution sensors. We
present a method whereby an robot in a continuous world can, with
little prior knowledge of its sensorimotor system, environment, and
task, improve task learning by first using a self-organizing feature
map to develop a set of higher-level perceptual features while
exploring using primitive, local actions. Then using those features,
the agent can build a set of high-level actions that carry it between
perceptually distinctive states in the environment. This method
combines a perceptual abstraction of the agent's sensory input
into useful perceptual features, and a temporal abstraction of the
agent's motor output into extended, high-level actions, thus
reducing both the dimensionality and the diameter of the task. An
experiment on a simulated robot navigation task shows that the agent
using this method can learn to perform a task requiring 300
small-scale, local actions using as few as 7 temporally-extended,
abstract actions, significantly improving learning time.},
}
Generated by bib2html.pl (written by Patrick Riley ) on Fri Nov 02, 2007 19:46:51