Feed-forward learning: Fast reinforcement learning of controllers

Abstract

Reinforcement Learning (RL) approaches are, very often, rendered useless by the statistics of the required sampling process. This paper shows how very fast RL is essentially made possible by abandoning the state feedback during training episodes. The resulting new method, feed-forward learning (FF learning), employs a return estimator for pairs of a state and a feed-forward policy's parameter vector. FF learning is particularly suitable for the learning of controllers, e.g. for robotics applications, and yields learning rates unprecedented in the RL context. This paper introduces the method formally and proves a lower bound on its performance. Practical results are provided from applying FF learning to several scenarios based on the collision avoidance behavior of a mobile robot. © Springer-Verlag Berlin Heidelberg 2007.