"a distributional perspective on reinforcement learning"
Abstract: This talk will present an overview of our recent research on distributional reinforcement learning. In a recent paper we argued for the fundamental importance of the value distribution: the distribution of random returns received by a reinforcement learning agent. This is in contrast to the common approach, which models the expectation of this return, or value. We identify distributional versions of Bellman's equations for both the policy evaluation and control settings, and develop the mathematical formalism needed to study the convergence of the corresponding iterative processes. We use the distributional perspective to design a new algorithm which learns the value distribution through a TD-like bootstrap process. We combine this algorithm with the DQN architecture and evaluate it on the suite of games from the Arcade Learning Environment (ALE). The resulting agent, called C51, outperforms state-of-the-art agents. This is particularly surprising given that C51 still acts to maximize expectations, and I conclude by discussing the hypotheses that might explain our algorithm's striking success.
Bio: Marc G. Bellemare received his Ph.D. from the University of Alberta, where he studied the concept of domain-independent agents and led the development of the highly-successful Arcade Learning Environment, the platform for AI research on Atari 2600 games. His interests include reinforcement learning, online learning, information theory, lifelong learning, and randomized algorithms. After four years at DeepMind, he recently joined Google Brain Montreal as a senior research scientist.
ai seminar series
Amii and the Department of Computing Science host AI Seminars, engaging presentations on topics in the broad field of artificial intelligence. With speakers from the University of Alberta and other world-leading groups, the talks give AI enthusiasts a friendly way of engaging with the latest trends and topics in research and development.
Seminars are open to the public, and no registration is required, though seating is limited and on a first-come-first-served basis. Topics range from foundational theoretical work to innovative applications of artificial intelligence technologies.
If you would like to present at an upcoming AI Seminar, please contact Colin Bellinger.
Join the AI Seminar mailing list to stay up-to-date on all the latest presentations.