Jefferson Provost, Benjamin J. Kuipers, and Risto Miikkulainen. Self-Organizing Distinctive State Abstraction Using Options. In Proceedings of the 7th International Conference on Epigenetic Robotics, 2007.
The behavior of a realistic robotic agent takes place in high-dimensional continuous sensory, state, and motor spaces. Autonomous learning of effective behaviors requires autonomous learning of useful abstractions for these spaces. The concept of distinctive state from the topological mapping literature can be used to learn such actions from the agent’s own experience, without prior knowledge provided by an external designer. In the approach taken in this paper, Self-Organizing Distinctive-state Abstraction (SODA), a variant of self-organizing maps defines a finite set of distinctive sensory prototypes; distinctive states are then defined as local maxima of the activation function for the leading prototype. Hierarchical reinforcement learning is then used to learn options that move the agent among distinctive states with increasing reliability. This state-action abstraction is learned autonomously, and reflects only the environment and the agent’s sensorimotor capabilities, without external direction. Using SODA, a robot can learn to navigate in large environments that are intractable to learn in using primitive motor commands.
@InProceedings{provost-epirob07,
author = {Jefferson Provost and Benjamin J. Kuipers and Risto Miikkulainen},
title = {Self-Organizing Distinctive State Abstraction Using Options},
booktitle = {Proceedings of the 7th International Conference on Epigenetic Robotics},
year = 2007,
volume = {7},
abstract = {The behavior of a realistic robotic agent takes place in
high-dimensional continuous sensory, state, and
motor spaces. Autonomous learning of effective
behaviors requires autonomous learning of useful
abstractions for these spaces. The concept of
distinctive state from the topological mapping
literature can be used to learn such actions from
the agent’s own experience, without prior knowledge
provided by an external designer. In the approach
taken in this paper, Self-Organizing
Distinctive-state Abstraction (SODA), a variant of
self-organizing maps defines a finite set of
distinctive sensory prototypes; distinctive states
are then defined as local maxima of the activation
function for the leading prototype. Hierarchical
reinforcement learning is then used to learn options
that move the agent among distinctive states with
increasing reliability. This state-action
abstraction is learned autonomously, and reflects
only the environment and the agent’s sensorimotor
capabilities, without external direction. Using
SODA, a robot can learn to navigate in large
environments that are intractable to learn in using
primitive motor commands.}
}
Generated by bib2html.pl (written by Patrick Riley ) on Fri Nov 02, 2007 19:46:51