This paper presents a set of methods by which a learning agent can learn a sequence of increasingly abstract and powerful interfaces to control a robot whose sensorimotor apparatus and environment are initially unknown. The result of the learning is a rich hierarchical model of the robot's world (its sensimotor apparatus and environment). The learning methods rely on generic properties of the robot's world such as almost-everywhere smooth effects of motor control signals on sensory features.
At the lowest level of the hierarchy, the learning agent analyzes the effects of its motor control signals in order to define a new set of control signals, one of each of the robot's degrees of freedom. It uses a generate-and-test approach to define sensory features that capture important aspects of the environment. It uses linear regression to learn models that characterize context-dependent effects of the control laws for finding and following paths defined using constraints on the learned features. The agent abstracts these control laws, which interact with the continuous environment, to a finite set of actions that implement discrete state transitions. At this point, the agent has abstracted the robot's continuous world to a finite-state world and can use existing methods to learn its structure.
The learning agent's methods are evaluated on several simulated robots with different sensorimotor systems and environments.
The Spatial Semantic Hierarchy is a model of knowledge of large-scale space consisting of multiple interacting representations, both qualitative and quantitative. The SSH is inspired by the properties of the human cognitive map, and is intended to serve both as a model of the human cognitive map and as a method for robot exploration and map-building. The multiple levels of the SSH express states of partial knowledge, and thus enable the human or robotic agent to deal robustly with uncertainty during both learning and problem-solving.
The control level represents useful patterns of sensorimotor interaction with the world in the form of trajectory-following and hill-climbing control laws leading to locally distinctive states. Local geometric maps in local frames of reference can be constructed at the control level to serve as observers for control laws in particular neighborhoods. The causal level abstracts continuous behavior among distinctive states into a discrete model consisting of states linked by actions. The topological level introduces the external ontology of places, paths and regions by abduction, to explain the observed pattern of states and actions at the causal level. Quantitative knowledge at the control, causal and topological levels supports a ``patchwork map'' of local geometric frames of reference linked by causal and topological connections. The patchwork map can be merged into a single global frame of reference at the metrical level when sufficient information and computational resources are available.
We describe the assumptions and guarantees behind the generality of the SSH across environments and sensorimotor systems. Evidence is presented from several partial implementations of the SSH on simulated and physical robots.
Research in Qualitative Reasoning builds and uses discrete symbolic models of the continuous world. Inference methods such as qualitative simulation are grounded in the theory of ordinary differential equations. We argue here that cognitive mapping --- building and using symbolic models of the large-scale spatial environment --- is a highly appropriate domain for qualitative reasoning research.
We describe the Spatial Semantic Hierarchy (SSH), a set of distinct representations for space, each with its own ontology, each with its own mathematical foundation, and each abstracted from the levels below it. At the control level, the robot and its environment are modeled as a continuous dynamical system, whose stable equilibrium points are abstracted to a discrete set of ``distinctive states.'' Trajectories linking these states can be abstracted to actions, giving a discrete causal graph level of representation for the state space. Depending on the properties of the actions, the causal graph can be deterministic or stochastic. The causal graph of states and actions can in turn be abstracted to a topological network of places and paths. Local metrical models, such as occupancy grids, of neighborhoods of places and paths can then be built on the framework of the topological network while avoiding their usual problems of global consistency.
This paper gives an overview of the SSH, describes the kinds of guarantees that the representation can support, and gives examples from two different robot implementations. We conclude with a brief discussion of the relation between the concepts of ``distinctive state'' and ``landmark value.''