News
The AI Seminar is a weekly meeting at the University of Alberta where researchers interested in artificial intelligence (AI) can share their research. Presenters include both local speakers from the University of Alberta and visitors from other institutions. Topics can be related in any way to artificial intelligence, from foundational theoretical work to innovative applications of AI techniques to new fields and problems.
On Feb. 17, Ehsan Imani— PhD student at the University of Alberta — presented “Representation Alignment in Neural Networks" at the AI Seminar.
It is now a standard for neural network representations to be trained on large, publicly available datasets, and used for new problems. The reasons why neural network representations have been so successful for transfer, however, are still not fully understood.
Imani shows that, after training, neural network representations align their top singular vectors to the targets. He investigates this representation alignment phenomenon in a variety of neural network architectures and finds that (a) alignment emerges across a variety of different architectures and optimizers, with more alignment arising from depth (b) alignment increases for layers closer to the output and (c) existing high-performance deep CNNs exhibit high levels of alignment.
The presentation then highlights why alignment between the top singular vectors and the targets can speed up learning and show in a classic synthetic transfer problem that representation alignment correlates with positive and negative transfer to similar and dissimilar tasks.
Watch the full presentation below:
Want to learn how you can kick-start your AI career? Find out more about Amii's Career Accelerator to find out more.
May 31st 2023
News
A new paper presented this week by Amii scientists at the International Conference on Autonomous Agents and Multiagent Systems (AAMAS) details how they could use machine learning to significantly increase the efficiency of Attabotic's robotic warehouses and the promise the technology holds to transform supply chain logistics.
May 31st 2023
News
On April 28, Jiamin He —MSc student at the University of Alberta — presented “Consistent Emphatic Temporal-Difference Learning" at the AI Seminar.
May 11th 2023
News
Russ Greiner — Amii Fellow, Canada CIFAR AI Chair and one of the founding members of Amii — is being recognized for his outstanding contribution to furthering computing science with a Lifetime Achievement Award in Computing Science from CS-Can | Info-Can.
Looking to build AI capacity? Need a speaker at your event?