Two research projects by Amii researchers that aim to increase the trustworthiness and security of advanced AI systems have received funding from CIFAR’s Canadian AI Safety Institute (CAISI) Research Program.
Amii Fellows and Canada CIFAR AI Chairs Linglong Kong and Bei Jiang are among four Canadian AI researchers chosen to have their projects funded as part of the AI Alignment Project, an international funding coalition led by the UK AI Security Institute. The project is a collaboration between governments, research organizations, and philanthropists supporting research into one of the field’s most pressing questions: how to make sure AI remains beneficial and secure for people.
The science of AI safety is absolutely critical. And Amii is well-positioned to lead in this area, focused on the core scientific and research questions that are foundational to making AI systems safe, reliable, and trustworthy.
Alyssa Lefaivre Škopac
Amii’s Director of AI Trust & Safety.
The collaboration between CAISI, the UK AI Security Institute, and industry and philanthropic partners, aligned on AI safety research, is encouraging, says Alyssa Lefaivre Škopac, Amii’s Director of AI Trust & Safety. And the inclusion of Kong and Jiang reflects the relevance and rigor of Amii’s researchers.“The science of AI safety is absolutely critical,” she says.
“And Amii is well-positioned to lead in this area, focused on the core scientific and research questions that are foundational to making AI systems safe, reliable, and trustworthy.
Both Kong and Jiang’s projects will receive $165,000 to fund their safety research over the next year, with the option to extend another year. They will also receive specialized computing resources and additional support to aid their research.
Learn more about the researchers and their projects
Linglong Kong
Kong’s research project will focus on the problem of “emergent misalignment” that often afflicts machine learning models that are deployed in the real world. A model that is originally built to align with a certain goal can, over time, learn behaviours that cause it to “cheat” or take undesirable shortcuts (called reward hacking) in tasks, or sometimes even move away from its original purpose without the user’s knowledge (deceptive alignment).
His research will test whether a statistical approach to online fine-tuning — where these behaviours are corrected with training in real-time — is effective in addressing problems with emergent alignment.
Bei Jiang
One of the great challenges in AI safety research is dealing with “long-tail failures,” rare and unexpected failures that happen so infrequently that they slip by ordinary testing. Identifying and mitigating these errors are crucial to building reliable, trustworthy AI systems.
Jiang’s project will explore new methods of quantifying these hard-to-spot errors in large language models by using statistical tools found in other fields that deal with rare events but are underused in the AI space.
In addition to the two Amii researchers, two other research projects by Canada CIFAR AI Chairs at Toronto’s Vector Institute also received funding under the CAISI collaboration. They are part of recent investments in strengthening research into safe and trustworthy AI, both from Amii and CIFAR.
“The AI Alignment Project is an important step in furthering Canada’s long history of leadership in driving AI research that is safe and trustworthy. As AI becomes increasingly present in our lives, it is more important than ever to ensure it is aligned with our values and serves the public good. By investing in the work of these Canadian researchers, we are building long-term economic resilience while cementing Canada’s position as a global leader in responsible AI development and deployment,” says Evan Solomon, Minister of Artificial Intelligence and Digital Innovation and Minister Responsible for the Federal Economic Development Agency for Southern Ontario.
Learn more about the milestones we reached in our Trust & Safety program at Amii in 2025.