News
The AI Seminar is a weekly meeting at the University of Alberta where researchers interested in artificial intelligence (AI) can share their research. Presenters include both local speakers from the University of Alberta and visitors from other institutions. Topics can be related in any way to artificial intelligence, from foundational theoretical work to innovative applications of AI techniques to new fields and problems.
On April 28, Jiamin He —MSc student at the University of Alberta — presented “Consistent Emphatic Temporal-Difference Learning" at the AI Seminar.
Off-policy policy evaluation has been a critical and challenging problem in reinforcement learning, and Temporal-Difference (TD) learning is one of the most important approaches for addressing it. There has been significant interest in searching for consistent off-policy TD algorithms that are guaranteed to find the on-policy TD fixed point. Notably, Full Importance-Sampling TD is the only existing consistent off-policy TD method under general linear function approximation but, unfortunately, has a high variance and is scarcely practical.
This notorious high variance issue motivates the introduction of Emphatic TD, which tames down the variance but has a biased fixed point. Inspired by these two methods, He proposes a new consistent algorithm called Average Emphatic TD (AETD) with a transient bias, which strikes a balance between bias and variance. Further, He unifes AETD with several existing algorithms and obtains a new family of consistent algorithms called Consistent Emphatic TD (CETD), which can control a smooth bias-variance trade-off by varying the speed at which the transient bias fades. Through theoretical analysis and experiments on a didactic example, He validates the consistency of CETD. Moreover, He shows that CETD converges faster to the lowest error in a complex task with a high variance.
Watch the full presentation below:
Want to learn how you can kick-start your AI career? Find out more about Amii's Career Accelerator to find out more.
Apr 8th 2024
News
Amii Fellows share tips on how to make the most of your conference experience.
Mar 26th 2024
News
In this month's episode, Alona talks about how ChatGPT changed the public’s perception of what AI language models can do, instantly making most previous benchmarks seem out of date, and the excitement and intensity of working in a fast-moving field like AI.
Mar 18th 2024
News
Google.org announces new research grants to support critical AI research in Canada focused on areas such as sustainability and the responsible development of AI. The grant will provide a total of $2.7 million in grant funding to Amii, the Canadian Institute for Advanced Research (CIFAR) and the International Center of Expertise of Montreal on AI (CEIMIA).
Looking to build AI capacity? Need a speaker at your event?