"sample-efficient reinforcement learning with rich observations"
Abstract: We study a version of reinforcement learning in which the agent must learn how to choose actions based on observations so as to maximize long-term reward. We focus especially on when the observations may be realistically rich, such as images, text documents, patient records, etc. We introduce a new algorithm for systematic exploration, in other words, for discovering through experimentation how best to choose actions. Along the way, we also propose a new measure called the “Bellman rank” which we argue captures the degree to which the learning problem exhibits underlying structure, and which can be favorably bounded in a number of previously studied cases. We show that the Bellman rank determines the statistical efficiency of our algorithm, which, although not computationally efficient, requires a number of samples that is polynomial in the Bellman rank as well as more standard parameters, but which is entirely independent of the size of the observation space.
This work is joint with Nan Jiang, Akshay Krishnamurthy, Alekh Agarwal, and John Langford.
Bio: Robert Schapire is a Principal Researcher at Microsoft Research in New York City. He received his PhD from MIT in 1991. After a short post-doc at Harvard, he joined the technical staff at AT&T Labs (formerly AT&T Bell Laboratories) in 1991. In 2002, he became a Professor of Computer Science at Princeton University. He joined Microsoft Research in 2014. His awards include the 1991 ACM Doctoral Dissertation Award, the 2003 Gödel Prize, and the 2004 Kanelakkis Theory and Practice Award (both of the last two with Yoav Freund). He is a fellow of the AAAI, and a member of both the National Academy of Engineering and the National Academy of Sciences. His main research interest is in theoretical and applied machine learning, with particular focus on boosting, online learning, game theory, and maximum entropy
ai seminar series
Fridays at noon, Amii and the Department of Computing Science host AI Seminars, engaging presentations on topics in the broad field of artificial intelligence. With speakers from the University of Alberta and other world-leading groups, the talks give AI enthusiasts a friendly way of engaging with the latest trends and topics in research and development.
Seminars are open to the public, and no registration is required, though seating is limited and on a first-come-first-served basis. Topics range from foundational theoretical work to innovative applications of artificial intelligence technologies.
If you would like to present at an upcoming AI Seminar, please contact Colin Bellinger.
Join the AI Seminar mailing list to stay up-to-date on all the latest presentations