The arcade learning environment: An evaluation platform for general agents

In this article we introduce the Arcade Learning Environment (ALE): both a challenge problem and a platform and methodology for evaluating the development of general, domain-independent AI technology. ALE provides an interface to hundreds of Atari 2600 game environments, each one different, interesting, and designed to be a challenge for human players. ALE presents significant research challenges for reinforcement learning, model learning, model-based planning, imitation learning, transfer learning, and intrinsic motivation. Most importantly, it provides a rigorous testbed for evaluating and comparing approaches to these problems. We illustrate the promise of ALE by developing and benchmarking domain-independent agents designed using well-established AI techniques for both reinforcement learning and planning. In doing so, we also propose an evaluation methodology made possible by ALE, reporting empirical results on over 55 different games. All of the software, including the benchmark agents, is publicly available.

Acknowledgements

We would like to thank Marc Lanctot, Erik Talvitie, and Matthew Hausknecht for providing suggestions on helping debug and improving the Arcade Learning Environment source code.We would also like to thank our reviewers for their helpful feedback and enthusiasm about the Atari 2600 as a research platform. The work presented here was supported by theAlberta Innovates Technology Futures, the Alberta Innovates Centre for Machine Learningat the University of Alberta, and the Natural Science and Engineering Research Council ofCanada. Invaluable computational resources were provided by Compute/Calcul Canada.