News
The AI Seminar is a weekly meeting at the University of Alberta where researchers interested in artificial intelligence (AI) can share their research. Presenters include both local speakers from the University of Alberta and visitors from other institutions. Topics can be related in any way to artificial intelligence, from foundational theoretical work to innovative applications of AI techniques to new fields and problems.
On May 12, Brett Daley —Phd student at the University of Alberta — presented “Trajectory-Aware Eligibility Traces for Off-Policy Reinforcement Learning" at the AI Seminar.
Off-policy learning from multistep returns is crucial for sample-efficient reinforcement learning, but counteracting off-policy bias without exacerbating variance is challenging. Classically, off-policy bias is corrected in a per-decision manner: past temporal-difference errors are re-weighted by the instantaneous Importance Sampling (IS) ratio after each action via eligibility traces. Many off-policy algorithms rely on this mechanism, along with differing protocols for cutting the IS ratios to combat the variance of the IS estimator. Unfortunately, once a trace has been fully cut, the effect cannot be reversed. This has led to the development of credit-assignment strategies that account for multiple past experiences at a time. These trajectory-aware methods have not been extensively analyzed, and their theoretical justification remains uncertain.
In this talk, Daley proposes a unifying framework for per-decision and trajectory-aware methods, and establishes the first general convergence conditions for trajectory awareness in the tabular setting. He also introduces a new algorithm called Recency-Bounded Importance Sampling (RBIS), which leverages trajectory awareness to perform robustly across hyperparameters in several off-policy control tasks.
Watch the full presentation below:
Want to learn how you can kick-start your AI career? Find out more about Amii's Career Accelerator to find out more.
Apr 8th 2024
News
Amii Fellows share tips on how to make the most of your conference experience.
Mar 26th 2024
News
In this month's episode, Alona talks about how ChatGPT changed the public’s perception of what AI language models can do, instantly making most previous benchmarks seem out of date, and the excitement and intensity of working in a fast-moving field like AI.
Mar 18th 2024
News
Google.org announces new research grants to support critical AI research in Canada focused on areas such as sustainability and the responsible development of AI. The grant will provide a total of $2.7 million in grant funding to Amii, the Canadian Institute for Advanced Research (CIFAR) and the International Center of Expertise of Montreal on AI (CEIMIA).
Looking to build AI capacity? Need a speaker at your event?