"exploiting the relationship between forward and inverse reinforcement learning"
Presenter: Dale Schuurmans, Professor, Department of Computing Science, University of Alberta; Principal Investigator, Amii; Principal Investigator, RLAI
Abstract: "I will discuss a new approach to structured output learning that exploits a simple connection between forward and inverse reinforcement learning.
The key observation is that these two problems can be expressed as minimizing an identical Bregman divergence but in opposite directions. The connection is not merely theoretical: it allows one to draw a precise relation between tempered log-likelihood and regularized expected reward, revealing that their difference is a simple variance term.
This observation suggests new approaches for exploiting supervised data to efficiently estimate the expected (regularized) reward of a given policy. We have been exploiting this connection to develop a principled yet practical structured output training method where standard supervised learning (maximum likelihood) is applied to edited target labellings that have been sampled proportionally to their exponentiated scaled reward. Recently, we have also been using this perspective to develop a principled unification of actor-critic methods."
Joint work with Mohammad Norouzi, with the assistance of Samy Bengio, Zhifeng Chen, Navdeep Jaitly, Mike Schuster, and Yonghui Wu.
ai seminar series
Fridays at noon, Amii and the Department of Computing Science host AI Seminars, engaging presentations on topics in the broad field of artificial intelligence. With speakers from the University of Alberta and other world-leading groups, the talks give AI enthusiasts a friendly way of engaging with the latest trends and topics in research and development.
Seminars are open to the public, and no registration is required, though seating is limited and on a first-come-first-served basis. Topics range from foundational theoretical work to innovative applications of artificial intelligence technologies.
If you would like to present at an upcoming AI Seminar, please contact Colin Bellinger.
Join the AI Seminar mailing list to stay up-to-date on all the latest presentations.