A Survey of Reinforcement Learning in Relational Domains

Reinforcement learning has developed into a primary approach for learning control strategies for autonomous agents. However, most of the work has focused on the algorithmic aspect, i.e. various ways of computing value functions and policies. Usually the representational aspects were limited to the use of attribute-value or propositional languages to describe states, actions etc. A recent direction - under the general name of relational reinforcement learning - is concerned with upgrading the representation of reinforcement learning methods to the first-order case, being able to speak, reason and learn about objects and relations between objects. This survey aims at presenting an introduction to this new field, starting from the classical reinforcement learning framework. We will describe the main motivations and challenges, and give a comprehensive survey of methods that have been proposed in the literature. The aim is to give a complete survey of the available literature, of the underlying motivations and of the implications if the new methods for learning in large, relational and probabilistic environments.