Towards Reinforcement Learning in the Continuing Setting

Abstract:

Many sequential decision making problems can be naturally formulated as continuing tasks in which the agent-environment interaction goes on forever without limit. In this paper we outline the state of research in the continuing setting. We trace the main results of the two alternative ways of framing a continuing problem—the discounted and the average-reward formulations. Unlike the episodic case, reinforcement learning (RL) solution methods for the continuing setting are not well understood, theoretically or empirically. We identify that RL research lacks a collection of easy-to-use continuing domains that can help foster our understanding of the problem setting and its solution methods. To stimulate research in the RL methods for the continuing setting, we finally sketch a preliminary set of continuing domains that we refer to as C-suite.

This paper is being presented at the Never-Ending Reinforcement Learning (NERL) workshop as part of the 2021 International Conference on Learning Representations (ICLR).