Community - Partner Event

RL Theory Seminar: Corruption robust exploration in episodic reinforcement learning

Nov. 3, 2020 - Nov. 3, 2020
11:00 AM - 12:00 PM


Amii is proud to support our province's growing AI community. The RL Theory Seminars are hosted independently by researchers: Gergely Neu, Ciara Pike-Burke, and Amii Fellow Csaba Szepesvári.

Speaker: Thodoris Lykouris (Microsoft Research)

Paper: https://arxiv.org/abs/1911.08689

Authors: Thodoris Lykouris, Max Simchowitz, Aleksandrs Slivkins, Wen Sun

Abstract: We initiate the study of multi-stage episodic reinforcement learning under adversarial corruptions in both the rewards and the transition probabilities of the underlying system extending recent results for the special case of stochastic bandits. We provide a framework which modifies the aggressive exploration enjoyed by existing reinforcement learning approaches based on "optimism in the face of uncertainty", by complementing them with principles from "action elimination". Importantly, our framework circumvents the major challenges posed by naively applying action elimination in the RL setting, as formalized by a lower bound we demonstrate. Our framework yields efficient algorithms which (a) attain near-optimal regret in the absence of corruptions and (b) adapt to unknown levels corruption, enjoying regret guarantees which degrade gracefully in the total corruption encountered. To showcase the generality of our approach, we derive results for both tabular settings (where states and actions are finite) as well as linear-function-approximation settings (where the dynamics and rewards admit a linear underlying representation). Notably, our work provides the first sublinear regret guarantee which accommodates any deviation from purely i.i.d. transitions in the bandit-feedback model for episodic reinforcement learning.

Related Events

Community - Partner Event

AI4Good Lab

May 3, 2021 - June 22, 2021
Community - Partner Event - Industry - Partner Event

Uniting the Prairies

Uniting the Prairies (UP 2021) is a virtual conference that brings together the top startups from across the Prairies to connect with global investors & tech leaders and collectively elevate the Prairie Tech Community.

June 23, 2021 - June 24, 2021


Community - Amii Event - Community - Partner Event

AI Strategy and Management - Spring 2021

May 31, 2021 - July 20, 2021
1:00 PM

Virtual Delivery

Connect with the community

Get involved in Alberta's growing AI ecosystem! Speaker, sponsorship, and letter of support requests welcome.

Explore training and advanced education

Curious about study options under one of our researchers? Want more information on training opportunities?

Harness the potential of artificial intelligence

Let us know about your goals and challenges for AI adoption in your business. Our Investments & Partnerships team will be in touch shortly!