Make Sense of Machine Learning

Heralded by many as the fourth industrial revolution, artificial intelligence has inspired countless news articles, novels, and films. With this deluge of information comes hopes and aspirations, fears and misconceptions – some justified and others not.

How can we make sense of it all? 

Join Amii as we pull back the curtain on AI and lay a pragmatic foundation for what AI is, what it isn’t, and how you can begin to think about the place of AI in your business.

Event Information

Register today!

Wednesday, December 5, 2018
5:15 p.m. – 6:30 p.m.

The Global Business Centre
136 8th Ave SE
Terrace View Room (5th floor)
Calgary, AB

Together with the University of Alberta (UAlberta) and Simon Fraser University, Amii is pleased to welcome four new Canada CIFAR AI (CCAI) Chairs to our family! Congratulations to Angel Chang, Alona Fyshe, Martha White and James Wright as they join a rapidly growing community of world-leading researchers in Canada.

Check out the following post from our friends at UAlberta’s Faculty of Science (with some additions from our team) and learn more about the brilliant researchers who are helping to drive the future of machine intelligence.

Building smarter computers and maximizing human capacity:

New artificial intelligence research chairs will push the field forward, with critical funding infusion

From helping humans make better decisions informed by machine intelligence models to building better computers using human brain models, the future has never looked brighter for AI’s influence in society. The slow but steady build of the critical mass of Edmonton’s artificial intelligence brainpower on the global scene has tipped into an explosion in Canadians’ consciousness.

So how do you draw some of the world’s most in-demand minds to a landlocked winter city and keep them here past the first blizzard?

Designed to attract and retain the brains behind the drive to integrate AI into sustainable solutions for our increasingly data-driven society, the first cohort of the Canada CIFAR AI Chairs was announced this morning in Montreal. The illustrious list includes four promising early-career researchers at Amii.

Meet the new chairs supported by Amii, three of whom are assistant professors in the UAlberta’s Faculty of Science, who will collectively receive roughly $3M over the next five years to support their research:

Creating brilliant computers using the human brain

Alona FysheWhat is the best way to improve computers? Study humans, said Alona Fyshe, assistant professor cross-appointed between the Departments of Computing Science and Psychology and Amii fellow. Fyshe applies her expertise in machine learning to brain imaging data, with the purpose of understanding how humans create meaning and use that meaning to make inferences about the world around them.

“What I try to figure out is how we go from words on a page, or pixels on a screen, to higher order meaning,” explained Fyshe. “Right now, computers aren’t very good at that. I’m interested in studying how people make meaning in order to improve how computers do the same thing.”

Fyshe, a UAlberta alumna, returns to her alma mater after time spent on faculty at the University of Victoria and Google. She completed graduate studies with University of Alberta professors Paul Lu, Duane Szafron, and renowned AI researcher and Amii fellow Russ Greiner, whose research drives progress in the field of precision health.

Fundamental research that’s anything but basic

Martha WhiteHow do you help humans make better decisions? You turn a bunch of raw data into complex models that generate accurate predictions. But how exactly do you tackle that technical task? Enter Martha White, Amii fellow and assistant professor in the Department of Computing Science, whose research is dedicated to fundamental algorithms using artificial intelligence and machine learning.

“It’s about supporting humans in making better decisions,” said White of artificial intelligence and machine learning. “AI isn’t scary. It’s just helpful. I’m excited about the answering the interesting questions we’ve had all along in AI. How do we get our agents to explore well, and how do we build good representations of the world? People are excited, which means we now have more access to students and to data and computational resources.”

White, also a graduate of UAlberta, completed her graduate studies with UAlberta professor and Amii fellow Michael Bowling, known for his landmark work with computer poker research and as a research scientist with DeepMind, as well as Dale Schuurmans (UAlberta professor and Amii fellow), a highly influential mind in the AI world. White returned to her alma mater following a professorial position at Indiana State University.

Using game theory to predict human behaviour

James WrightWhy would you look to game theory to better understand how humans make choices? That is the question driving James Wright’s research, one that sees him constructing data-driven models to improve the effectiveness of real systems by accurately predicting human decision making.

“Agents that interact with people in strategic systems will do a much better job if they can predict how people will react,” said Wright, assistant professor in the Department of Computing Science. “Similarly, a policy will achieve its goals more effectively when it takes into account how people will respond to policy. When new policies have unintended consequences, it’s often because they were designed without thinking about these kinds of strategic questions.”

Wright returned to Canada following time spent as a postdoctoral researcher at the Microsoft Research Lab in New York City, joining the computing scientists at UAlberta and contributing to Edmonton’s world-class power on the AI stage.

Developing a common understanding

Angel ChangHow can we help humans and computers better work together? If you’re Angel Chang, CCAI Chair at Amii and incoming assistant professor in the School of Computing Science at Simon Fraser University, you develop ways of using language to help computers understand and interact with our everyday world.

“I’m interested in developing AI that has a more robust and human-relatable understanding of our world,” says Chang. “My work – at the intersection of natural language understanding, computer graphics and AI – explores the representation and acquisition of common sense knowledge and how the semantics of shapes and scenes can be used to connect language to visual and 3D representations of the world.”

Chang, who received her PhD in Computer Science from Stanford University under the supervision of Chris Manning, will be joining Simon Fraser University in 2019. She is currently a research scientist at Eloquent Labs working on developing conversational AI that can understand and assist people. Previously, she was a postdoctoral scholar at Princeton University where she developed large-scale 3D datasets for deep learning and 3D scene understanding.

In 2017, CIFAR was chosen by the federal government to lead the $125M Pan-Canadian Artificial Intelligence Strategy in collaboration with artificial intelligence research centres in Edmonton (Amii), Montreal (Mila), and Toronto (Vector Institute). Support for these new chairs—including 29 across Canada—is part of a larger strategy including training opportunities, research funding, and workshops on the societal implications of AI designed to build on Canadian leadership in artificial intelligence.

Edmonton’s history of global AI dominance

The University of Alberta launched Canada’s first computing science department, dating back to 1964. Recent events—including the announcement of DeepMind’s first international research laboratory—have truly cemented Edmonton’s excellence on the global map. According to the acclaimed CS Rankings, UAlberta ranks third in the world for artificial intelligence and machine learning research.

Amii was founded in 2002 as a joint effort between UAlberta and the Government of Alberta with the goal of creating a world-class machine intelligence research centre. The organization has since spun out from UAlberta, while maintaining a strong partnership, with support from Alberta Innovates, the Government of Alberta and CIFAR—in order to drive new levels of discovery and innovation in AI and machine learning.

Make Sense of Machine Learning

Heralded by many as the fourth industrial revolution, artificial intelligence has inspired countless news articles, novels, and films. With this deluge of information comes hopes and aspirations, fears and misconceptions – some justified and others not.

How can we make sense of it all? 

Join Amii as we pull back the curtain on AI and lay a pragmatic foundation for what AI is, what it isn’t, and how you can begin to think about the place of AI in your business.

Event Information

Register today!

Monday, December 17, 2018
12 p.m. – 1 p.m.

Startup Edmonton
#301, 10359 104 Street Northwest
Edmonton, Alberta

Make Sense of Machine Learning

Heralded by many as the fourth industrial revolution, artificial intelligence has inspired countless news articles, novels, and films. With this deluge of information comes hopes and aspirations, fears and misconceptions – some justified and others not.

How can we make sense of it all? 

Join Amii as we pull back the curtain on AI and lay a pragmatic foundation for what AI is, what it isn’t, and how you can begin to think about the place of AI in your business.

Register today!

The Board of Directors of the Alberta Machine Intelligence Institute (Amii) announced Thursday the appointment of John Shillington as the institute’s first CEO. His appointment began on October 1, 2018.

As head of one Canada’s three AI institutes, Shillington will focus on further developing Alberta as a premier global destination for both foundational research and innovative commercial applications of artificial intelligence (AI) and machine learning technologies.

“With years of experience at the intersection of industry and academia, John is perfectly placed to lead Amii in growing Alberta into the world’s leading destination for AI and machine learning,” says Bruce Johnson, Chair of Amii’s Board of Directors. “He has an established record as a leader and builder of high-performing teams; he has in-depth knowledge of technology commercialization; and he is passionate about building a bright, AI-enabled future right here in Alberta.”

Shillington will work to boost Alberta’s global AI leadership through Amii’s four key focus areas: supporting world-class research and training at the University of Alberta (UAlberta), growing AI capacity in Alberta-based companies, attracting corporate research labs and upskilling Alberta’s workforce for AI literacy.

“I’m beyond excited to be joining such an amazing team of world-class researchers, staff and students,” says Shillington. “I look forward to leading the next stage of Amii’s growth as a not-for-profit, helping Alberta businesses realize the transformative potential of AI and accelerating our province’s global economic advantage.”

Shillington brings with him over 30 years of expertise as a technology leader with deep experience helping companies improve their competitive advantage and driving their next stage of growth. Much of his career has focused on helping organizations bridge the chasm between academic research and its applications in the public, private and not-for-profit sectors.

Elissa Strome, Executive Director of CIFAR’s Pan-Canadian AI Strategy echoes Johnson’s praise of Shillington: “We’re thrilled at the selection of John Shillington as leader of the Alberta Machine Intelligence Institute. Amii is already one of Canada’s AI research powerhouses, and we’re confident that John’s years of experience in the tech sector will provide the knowledge and vision needed to help further cement Canada’s reputation as one of the world’s premier locations for AI research and investment.”

Prior to joining Amii, Shillington was Vice President, Technology at Cybera, a not-for-profit technical agency helping Alberta advance its IT frontiers. He has also held senior executive and technical management positions at several international technology firms in addition to starting up a UAlberta-based spin-off company.


Adaptive Prosthetics Program

Principal Investigators
Patrick M. Pilarski, Richard S. Sutton

Intelligent Artificial Limbs & Biomedical Devices

The Adaptive Prosthetics Program, a collaboration between Amii and the BLINC Lab, is an interdisciplinary initiative focused on real-time machine learning methods for assistive rehabilitation and intelligent artificial limbs. Through the development of fundamental algorithms and the translation of methodology into clinical benefit, the program seeks to increase patients’ ability to customize and control assistive biomedical devices.

The Adaptive Prosthetics Program explores fundamental and applied methods for real-time prediction, adaptive control and direct human-machine interaction. Technologies developed through the program include the Bento Arm and the HANDi Hand, both of which have open-sourced hardware and software through the BLINCdev community.

See Also

Development of the Bento Arm: an Improved Robotic Arm for Myoelectric Training and Research” published at MEC ’14: Myoelectric Controls Symposium

Development of the HANDi Hand: an Inexpensive, Multi-Articulating, Sensorized Hand for Machine Learning Research in Myoelectric Control” published at MEC ’17: Myoelectric Controls Symposium


Arcade Learning Environment

Principal Investigator
Michael Bowling

Atari 2600 Platform for General Artificial Intelligence Development

The Arcade Learning Environment (ALE) is a software framework designed to facilitate the development and testing of general AI agents. ALE was created as both a challenge problem for AI researchers and a method for developing and evaluating innovations and technologies.

Historically, many AI advancements have been developed and tested through games (e.g. CheckersChessPoker and, most recently, Go), which offer controlled, well-understood environments with easily defined measures for success. Games also give researchers a concrete and relatable way to demonstrate artificial intelligence to a broad audience.

The Arcade Learning Environment, powered by the Stella Atari Emulator, provides an interface to hundreds of Atari 2600 games. These diverse game environments are complex enough to be challenging for an AI agent to solve yet simple enough to enable progress.

ALE is available to researchers and hobbyists alike with Atari now being used by groups like Google DeepMind to develop and test their deep reinforcement learning methodologies.

See Also

The Arcade Learning Environment: An Evaluation Platform for General Agents” published in the Journal of Artificial Intelligence Research



Principal Investigators
Randy Goebel, Osmar Zaïane

Social Network Analysis & Visualization

Meerkat is an automated Social Network Analysis (SNA) tool used to analyze, visualize and interpret large or complex networks of information, allowing users to examine patterns and investigate relational dynamics.

The application uses information about the interactions between a set of objects (or nodes) within a network and lets the user employ different algorithms to automatically identify meaningful connections or highlight the most influential or central nodes in different ways.

Network analysis features include:

  • Automated community detection and analysis
  • Interactive visualization using general, community and metric-based layouts
  • Filtration and extraction of useful data
  • Dynamic analysis of network changes over time

Meerkat also provides tools for text mining, including polarity and emotion analysis, which give users the ability to examine text for positive and negative sentiments and a range of basic emotions. Meerkat ED, a version of the program that has been tailored specifically for educational environments, allows instructors to evaluate student activities in online discussion forums.



Principal Investigator
Michael Bowling

Problem we’re trying to solve

For several years, AI researchers have had a number of different techniques for predicting and planning optimal actions in situations of perfect information (where all actors have the same, full knowledge of the world). Techniques have been lacking for dealing with imperfect information situations (where actors do not have access to certain information or have access to information the other doesn’t). DeepStack seeks to successfully apply, for the first time, theoretical techniques for perfect information games into situations with imperfect information.

How will this help someone / an industry?

For computing scientists and AI researchers, DeepStack represents a foundational step forward in dealing with issues around predicting optimal actions in the face of ambiguity and uncertainty. The theoretical advancements demonstrated in DeepStack will open new avenues of research for scientists interested in building, and planning with, models of unknown, complex dynamic systems.

Type of MI used

Reinforcement learning, Deep learning

(Amii’s Russ Greiner is part of a team of researchers from the University of Alberta and IBM who are using machine learning to help predict  schizophrenia .)

Pioneering research in “computational psychiatry” uses AI to explore disease prediction and assessment

IBM (NYSE: IBM) scientists and the University of Alberta in Edmonton, Canada, have published new data in Nature‘s partner journal, Schizophrenia, demonstrating that AI and machine learning algorithms helped predict instances of schizophrenia with 74% accuracy. This retrospective analysis also showed the technology predicted the severity of specific symptoms in schizophrenia patients with significant correlation, based on correlations between activity observed across different regions of the brain. This pioneering research could also help scientists identify more reliable objective neuroimaging biomarkers that could be used to predict schizophrenia and its severity.

Schizophrenia is a chronic and debilitating neurological disorder that affects 7 or 8 out of every 1,000 people. Those with schizophrenia can experience hallucinations, delusions or thought disorders, along with cognitive impairments, such as an inability to pay attention and physical impairments, such as movement disorders.

“This unique, innovative multidisciplinary approach opens new insights and advances our understanding of the neurobiology of schizophrenia, which may help to improve the treatment and management of the disease,” says Dr. Serdar Dursun, a Professor of Psychiatry & Neuroscience with the University of Alberta. “We’ve discovered a number of significant abnormal connections in the brain that can be explored in future studies, and AI-created models bring us one step closer to finding objective neuroimaging-based patterns that are diagnostic and prognostic markers of schizophrenia.”

In the paper, researchers analyzed de-identified brain functional Magnetic Resonance Imaging (fMRI) data from the open data set, Function Biomedical Informatics Research Network (fBIRN) for patients with schizophrenia and schizoaffective disorders, as well as a healthy control group. fMRI measures brain activity through blood flow changes in particular areas of the brain. Specifically, the fBIRN data set reflects research done on brain networks at different levels of resolution, from data gathered while study participants conducted a common auditory test. Examining scans from 95 participants, researchers used machine learning techniques to develop a model of schizophrenia that identifies the connections in the brain most associated with the illness.

Regions of the brain that showed a statistically significant difference between patients with schizophrenia and patients without it. (Arrow 1 identifies the precentral gyrus, or the motor cortex, and arrow 5 marks the precuneus, which involves processing visual information.)

The results of the IBM and University of Alberta research demonstrated that, even on more challenging neuroimaging data collected from multiple sites (different machines, across different groups of subjects etc.) the machine learning algorithm was able to discriminate between patients with schizophrenia and the control group with 74% accuracy using the correlations in activity across different areas of the brain.

Additionally, the research showed that functional network connectivity could also help determine the severity of several symptoms after they have manifested in the patient, including inattentiveness, bizarre behavior and formal thought disorder, as well as alogia, (poverty of speech) and lack of motivation. The prediction of symptom severity could lead to a more quantitative, measurement-based characterization of schizophrenia; viewing the disease on a spectrum, as opposed to a binary label of diagnosis or non-diagnosis. This objective, data-driven approach to severity analysis could eventually help clinicians identify treatment plans that are customized to the individual.

“The ultimate goal of this research effort is to identify and develop objective, data-driven measures for characterizing mental states, and apply them to psychiatric and neurological disorders” said Ajay Royyuru, Vice President of Healthcare & Life Sciences, IBM Research. “We also hope to offer new insights into how AI and machine learning can be used to analyze psychiatric and neurological disorders to aid psychiatrists in their assessment and treatment of patients.”

The Research Domain Criteria (RDoC) initiative of NIMH emphasizes the importance of objective measurements in psychiatry. This field, often referred to as “computational psychiatry”, aims to use modern technology and data driven approaches to improve evidence-based medical decision making in psychiatry, a field that often relies upon subjective evaluation approaches.

As part of the ongoing partnership, researchers will continue to investigate areas and connections in the brain that hold significant links to schizophrenia. Work will continue on improving the algorithms by conducting machine learning analysis on larger datasets, and by exploring ways to extend these techniques to other psychiatric disorders such as depression or post-traumatic stress disorder.

UAlberta Expertise Brings DeepMind Lab to Edmonton

In an historic move for the AI community, one of the world’s leading AI research companies, DeepMind, will open its first international research base outside the United Kingdom later this month. The lab will be based in Edmonton and have close ties to the University of Alberta, a research-intensive university with an illustrious record of AI research excellence.

The new lab, to be called DeepMind Alberta, demonstrates DeepMind’s commitment to accelerating Alberta’s and Canada’s AI research community. It also signals the strength of ties between the University of Alberta and one of the world’s leading AI companies. Having been acquired by Google in 2014, DeepMind is now part of Alphabet. DeepMind is on a scientific mission to push the boundaries of AI, developing programs that can learn to solve complex problems without being taught how. DeepMind Alberta will open with 10 employees.

The DeepMind Alberta team will be led by UAlberta computing science professors Richard Sutton, Michael Bowling, and Patrick Pilarski. All three, who will remain with the Alberta Machine Intelligence Institute at UAlberta, will also continue teaching and supervising graduate students at the university to further foster the Canadian AI talent pipeline and grow the country’s technology ecosystem. The team will be completed by seven more researchers, many of whom were also authors on the influential DeepStack paper published earlier this year in Science.

UAlberta’s connections to DeepMind run deep with roughly a dozen UAlberta alumni already working at the company, some of whom played important roles in some of DeepMind’s signature advances with reinforcement learning in AlphaGo and Atari. In addition, one of the world’s most renowned computing scientists, Sutton was DeepMind’s first advisor when the company was just a handful of people.

“I first met with Rich—our first ever advisor—seven years ago when DeepMind was just a handful of people with a big idea. He saw our potential and encouraged us from day one. So when we chose to set up our first international AI research office, the obvious choice was his base in Edmonton, in close collaboration with the University of Alberta, which has become a leader in reinforcement learning research thanks to his pioneering work,” said Demis Hassabis, CEO and co-founder of DeepMind. “I am very excited to be working with Rich, Mike, Patrick and their team, together with UAlberta, and I look forward to us making many more scientific breakthroughs together in the years ahead.”

Sutton is excited about the opportunity to combine the strength of DeepMind’s work in reinforcement learning with UAlberta’s academic excellence, all without having to leave Edmonton.

“DeepMind has taken this reinforcement learning approach right from the very beginning, and the University of Alberta is the world’s academic leader in reinforcement learning, so it’s very natural that we should work together,” said Sutton. “And as a bonus, we get to do it without moving.”

Working with Hassabis and the DeepMind team both in London and Edmonton, Sutton, Bowling, and Pilarski will combine their staggering academic strength in reinforcement learning to focus on basic AI research. Reinforcement learning functions similarly to the same way humans learn, trying to replicate good outcomes and avoid bad outcomes based on learned experiences.

The DeepMind Alberta announcement is the latest in a slate of AI-related successes for UAlberta. The recent major funding infusion via the federal government’s Pan-Canadian Artificial Intelligence Strategy strengthens the Alberta government’s 15-year investment of more than $40 million. DeepMind Alberta is a further signal that industry is taking notice of UAlberta and its boundary-pushing research.

About the Researchers

A professor in the Department of Computing Science in the University of Alberta’s Faculty of Science, Michael Bowling is best known for his research in poker, most notably with two milestone discoveries, both published in Science, Cepheus in 2015, which solved heads-up limit Texas hold’em followed by DeepStack in late 2016, which achieves professional-level play in heads-up no limit Texas hold’em.

Patrick Pilarski is the Canada Research Chair in Machine Intelligence for Rehabilitation and an assistant professor in the Department of Medicine (Division of Physical Medicine and Rehabilitation). His research interests include reinforcement learning, real-time machine learning, human-machine interaction, rehabilitation technology, and assistive robotics.

A professor in the Department of Computing Science in the University of Alberta’s Faculty of Science, Richard Sutton is world-renowned for his foundational research in reinforcement learning –he literally wrote the textbook–in which machines learn based on their environment. His landmark work has developed the area of temporal difference learning, which uses the future as a source of information for predictions, and also explores off-policy learning, or learning from actions not taken.

University of Alberta computing science professors and artificial intelligence researchers (L to R) Richard Sutton, Michael Bowling, and Patrick Pilarski are working with DeepMind to open the AI powerhouse company’s first research lab outside the United Kingdom in Edmonton, Canada.
Credit: John Ulan

UAlberta to Push Critical Areas of Research for the Future of All Canadians

EDMONTON (March 23, 2017)—In a move that will boost artificial intelligence (AI) research across the country, the Government of Canada announced Wednesday the funding of a pan-Canadian AI Strategy to enhance research and recruit talent. Administered through the Canadian Institute for Advanced Research (CIFAR), the $125 million program will promote collaboration between post-secondary institutions in Montréal, Toronto-Waterloo, and Edmonton.

“Our University of Alberta researchers have been world leaders in artificial intelligence for decades. I’m very pleased the government has moved forward and invested in this absolutely critical area for the future of all Canadians,” says University of Alberta President David Turpin.

With applications as diverse as enhanced medical diagnoses to self-driving cars, artificial intelligence is a continually growing area of research with the potential to transform all facets of society. The field already attracts investment from major technology players like Google, Facebook and Amazon. The global market for artificial intelligence-related products is expected to reach $47 billion by 2020.

“Canadian universities train some of the best AI researchers in the world,” says Cameron Schuler, Executive Director of the Alberta Machine Intelligence Institute (Amii), housed at the University of Alberta. “With this latest investment, Canada will build on existing strengths to retain and attract talented individuals, drive innovation and advances in industry, and take our place on the global stage as leaders in AI.”

The strategy aims to further develop Canada’s AI ecosystem and position the country as a world-leading destination for companies seeking to invest in artificial intelligence and innovation. “This investment in deep AI builds on Canada’s strength as a pioneer in AI research and will provide a strong foundation for Canada to build on its global leadership in this important and exciting field,” says Alan Bernstein, President and CEO of CIFAR.

In addition to retaining top talent, enhancing recruitment and training across Canada, the new funding will enable further collaborations between industry and academic institutions. Some of the recent industry collaborations at the University of Alberta include research partnerships, with large companies like RBC, and project-based ones, such as optimizing control systems with Edmonton-based companies like ISL Engineering and Willowglen Systems.

“There is some truly fascinating AI research coming out of Canada,” says Richard Sutton, professor of computing science in the University of Alberta’s Faculty of Science and researcher at Amii, world-renowned for his boundary pushing research in reinforcement learning. “Canada is punching above its weight in the field, and we’re thrilled the federal government is committed to building on this strong base. We’ve only scratched the surface of what AI can do and are excited to unleash even greater possibilities in deep reinforcement learning.”

Skill Trumps Luck: DeepStack the First Computer Program to Outplay Human Professionals at Heads-Up No-Limit Texas Hold’em Poker

EDMONTON (March 2, 2017)—A team of computing scientists from the University of Alberta’s Computer Poker Research Group is once again capturing the world’s collective fascination with artificial intelligence. In a historic result for the flourishing AI research community, the team—which includes researchers from Charles University and Czech Technical University in Prague—has developed an AI system called DeepStack that defeated professional poker players in December 2016.  The landmark findings have just been published in Science, one of the world’s most prestigious peer-reviewed scientific journals.

DeepStack bridges the gap between approaches used for games of perfect information—like those used in checkers, chess, and Go—with those used for imperfect information games, reasoning while it plays using “intuition” honed through deep learning to reassess its strategy with each decision.

“Poker has been a longstanding challenge problem in artificial intelligence,” says Michael Bowling, professor in the University of Alberta’s Faculty of Science and principal investigator on the study. “It is the quintessential game of imperfect information in the sense that the players don’t have the same information or share the same perspective while they’re playing.”

Don’t let the name fool you: imperfect information games are serious business. These “games” are a general mathematical model that describe how decision-makers interact. Artificial intelligence research has a storied history of using parlour games to study these models, but attention has been focused primarily on perfect information games. “We need new AI techniques that can handle cases where decision-makers have different perspectives,” says Bowling, explaining that developing techniques to solve imperfect information games will have applications well beyond the poker table.

“Think of any real world problem. We all have a slightly different perspective of what’s going on, much like each player only knowing their own cards in a game of poker.” Immediate applications include making robust medical treatment recommendations, strategic defense planning, and negotiation.

This latest discovery builds on an already impressive body of research findings about artificial intelligence and imperfect information games that stretches back to the creation of the University of Alberta’s Computer Poker Research Group in 1996. Bowling, who became the group’s principal investigator in 2006, has led the group to several milestones for artificial intelligence. He and his colleagues developed Polaris in 2008, beating top poker players at heads-up limit Texas hold’em poker. They then went on to solve heads-up limit hold’em with Cepheus, published in 2015 in Science.

DeepStack extends the ability to think about each situation during play—which has been famously successful in games like checkers, chess, and Go—to imperfect information games using a technique called continual re-solving. This allows DeepStack to determine the correct strategy for a particular poker situation without thinking about the entire game by using its “intuition” to evaluate how the game might play out in the near future.

“We train our system to learn the value of situations,” says Bowling. “Each situation itself is a mini poker game. Instead of solving one big poker game, it solves millions of these little poker games, each one helping the system to refine its intuition of how the game of poker works.  And this intuition is the fuel behind how DeepStack plays the full game.”

Thinking about each situation as it arises is important for complex problems like heads-up no-limit hold’em, which has vastly more unique situations than there are atoms in the universe, largely due to players’ ability to wager different amounts including the dramatic “all-in.” Despite the game’s complexity, DeepStack takes action at human speed—with an average of only three seconds of “thinking” time—and can run on a simple gaming laptop using an Nvidia GPU for computation.

To test the approach, DeepStack played against a pool of professional poker players in December, 2016, recruited by the International Federation of Poker. Thirty-three players from 17 countries were recruited, with each asked to play a 3000-hand match over a period of four weeks. DeepStack beat each of the 11 players who finished their match, with only one outside the margin of statistical significance, making it the first computer program to beat professional players in heads-up no-limit Texas hold’em poker.

“DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker” was published online by the journal Science on Thursday, March 2, 2017.

TORONTO, January 18, 2017 — Following recent investments in artificial intelligence (AI) and machine learning, RBC today announced Dr. Richard S. Sutton, one of the modern day pioneers of AI, as head academic advisor to RBC Research in machine learning. RBC Research will establish a new lab and plan to work with the Alberta Machine Intelligence Institute (Amii), based at the University of Alberta, to identify and pursue further research collaboration opportunities on an ongoing basis.

“We are thrilled to be opening a lab in Edmonton and to collaborate with world-class scientists like Dr. Sutton and the other researchers at Amii,” said Dr. Foteini Agrafioti, head of RBC Research. “RBC Research has built strong capabilities in deep-learning, and with this expansion, we are well poised to play a major role in advancing research in AI and impact the future of banking.”

Dr. Sutton is widely recognized for his work in reinforcement learning, an area of machine learning that focuses on making predictions without historical data or explicit examples. Reinforcement learning techniques have been shown to be particularly powerful in determining ideal behaviours in complex environments. Most recently, the techniques were used to secure a first-ever victory over a human world-champion in the game of Go, as well as recent applications in robotics and self-driving cars.

“The collaboration between RBC Research and Amii will help support the development of an AI ecosystem in Canada that will push the boundaries of academic knowledge,” said Dr. Sutton. “With RBC’s continued support, we will cultivate the next generation of computer scientists who will develop innovative solutions to the toughest challenges facing Canada and beyond. We’ve only scratched the surface of what reinforcement learning can do in finance and are excited to unleash even greater possibilities with this collaboration between RBC Research and Amii.”

“RBC is committed to helping build Canada’s digital future and our significant investments in AI represent part of that commitment,” said Gabriel Woo, vice-president of innovation at RBC. “We believe AI has the potential to bring about major improvements in areas such as client service, fraud prevention and risk management; advancements that will have far-reaching benefits in financial services and beyond. Partnering with a leading institution like the University of Alberta is an important step forward as we continue to explore this emerging technology.”

RBC Research is also collaborating with the University of Alberta to provide opportunities like internships, academic collaborations and exchanges with the Toronto-based research team to students and researchers. Dr. Eirene Seiradaki, academic partnerships lead at RBC, will be the key contact between RBC Research and professors, researchers and students interested in using machine learning to drive innovation in banking. With almost 20 years of experience in academics, Dr. Seiradaki joined RBC in 2016 and brings a strong commitment to fostering innovation and supporting the academic community.

RBC recently announced two additional initiatives in collaboration with the University of Toronto, ensuring Canada remains a leading centre of development in machine learning and AI.

Part of the Alberta Machine Intelligence Institute, Marlos C. Machado is a 4th year Ph.D. student in the University of Alberta’s Department of Computing Science, supervised by Amii’s Michael Bowling.

Marlos’ research interests lie broadly in artificial intelligence with a particular focus on machine learning and reinforcement learning. Marlos is also a member of the Reinforcement Learning & Artificial Intelligence research group, led by Amii’s Richard S. Sutton.

In 2013, Amii researchers proposed the Arcade Learning Environment (ALE), a framework that poses the problem of general competency in AI. The ALE allows researchers and hobbyists to evaluate artificial intelligence (AI) agents in a variety of Atari games, encouraging agents to succeed without game-specific information. While this may not seem like a difficult feat, up to now, intelligent agents have excelled at performing a single task at a time, such as checkers, chess and backgammon – all incredible achievements!

The ALE, instead, asks the AI to perform well at many different tasks: repelling aliens, catching fish and racing cars, among others. Around 2011, Amii’s Michael Bowling began advocating in the AI research community for an Atari-based testbed and challenge problem. The community has since recognized the importance of arcade environments, shown by the release of other, similar platforms such as the GVG-AI, the OpenAI Gym & Universe,  as well as the Retro Learning Environment.

Atari 2600 games
1. Atari 2600 games: Space Invaders, Bowling, Fishing Derby and Enduro

The ALE owes some of its success to a Google DeepMind algorithm called Deep Q-Networks (DQN), which recently drew world-wide attention to the learning environment and to reinforcement learning (RL) in general. DQN was the first algorithm to achieve human-level control in the ALE.

In this post, adapted from our paper, “State of the Art Control of Atari Games Using Shallow Reinforcement Learning,” published earlier this year, we examine the principles underlying DQN’s impressive performance by introducing a fixed linear representation that achieves DQN-level performance in the ALE.

The steps we took while developing this representation illuminate the importance of biases being encoded in neural networks’ architectures, which improved our understanding of deep reinforcement learning methods. Our representation also frees agents from necessarily having to learn representations every time an AI is evaluated in the ALE. Researchers can now use a good fixed representation while exploring other questions, which allows for better evaluation of the impact of their algorithms because the interaction with representation learning solutions can be put aside.

Impact of Deep Q-Networks

In reinforcement learning, agents must estimate how “good” a situation is based on current observations. Traditionally, we humans have had to define in advance how an agent processes the input stream based on the features we think are informative. These features can include anything from the position and velocity of an autonomous vehicle to the pixel values the agent sees in the ALE.

Before DQN, pixel values were frequently used to train AI in the ALE. Agents learned crude bits of knowledge like “when a yellow pixel appears on the bottom of the screen, going right is good.”  While useful, knowledge represented in this way cannot encode certain pieces of information such as in-game objects.

Because the goal of the ALE is to avoid extracting information particular to a single game, researchers faced the challenge of determining how an AI can succeed in multiple games without providing it game-specific information. To meet this challenge, the agent should not only learn how to act but also learn useful representations of the world.

DQN was one of the first RL algorithms capable of doing so with deep neural networks.

For our discussion, the important aspect of DQN is that its performance is due to the neural network’s estimation of how “good” each screen is, in other words how likely it is that a particular screen will result in a favourable outcome.

Importantly, the neural network has several convolutional layers with the ability to learn powerful internal representations. The layers are built around simple architectural biases such as position/translation invariance and the size of the filters used. We asked ourselves how much of DQN’s performance results from the internal representations learned and how much from the algorithm’s network architecture. We implemented, in a fixed linear representation, the biases encoded in DQN’s architecture and analyzed the gap between our bias-encoded performance and DQN’s performance.

To our surprise, our fixed linear representation performed nearly as well as DQN!

Basic & Blob-Prost Features

To create our representation, we first needed to define its building blocks. We used the method mentioned earlier of representing screens as “there is a yellow pixel at the bottom of the screen.”

As Figure 2 (inspired by the original article on the ALE) indicates, screens were previously defined in terms of the existence of colours in specific patches of the image. Researchers would divide the image in 14×16 patches and, for each patch, encode the colours available in that tile.

Screenshot and basic features of the game Space Invaders
2. Left: Screenshot of the game Space Invaders; Centre: Tiling used in all games; Right: Representation of Basic Features

In this example, two colours are present in the tile in the top left corner of the screen: black and green. Thus, the agent sees the whole tile as black and green with the “amount” of each colour being unimportant. This representation, called Basic, was introduced in the original paper on the ALE. However, Basic features don’t encode the relation between tiles, that is, “a green pixel is above a yellow pixel.” BASS features, which are not discussed in this post, can be used as a fix but with less than satisfactory results.

When DQN was proposed, it outperformed the state-of-the-art in the vast majority of games. But the question still remained: why?

One of our first insights was that convolutional networks apply the same filter in all different patches of the image, meaning observations aren’t necessarily encoded for a specific patch. In other words, instead of knowing “there is a green pixel in tile 6 and an orange pixel in tile 8,” the network knows “there is a green pixel one tile away from an orange pixel somewhere on the screen.”

This knowledge is useful as we no longer need to observe events at specific locations and can generalize them at the moment they occur. That is, the agent doesn’t need to be hit by an alien projectile in every possible pixel space to learn it’s bad. The AI quickly learns “a pixel above the green pixel (the player’s ship) is bad”, no matter the screen position. We modified Basic features to also encode such information, calling the new representation B-PROS.

Representation of B-PROS features
3. Representation of B-PROS features

B-PROS is limited in that it doesn’t encode objects movement. If there is a projectile on the screen, is it moving upwards from the agent’s ship or downwards from an alien’s?

We can easily answer the question by using two consecutive screens to infer an object’s direction, which is what DQN does. Instead of only using offsets from the same screen, we also looked at the offsets between different screens, encoding things like: “there was a yellow pixel two blocks above where the green pixel is now.” We call this representation B-PROST.

Representation of B-PROST features
4. Representation of B-PROST features

Finally, as is the case with DQN, we needed a way to identify objects. The filter sizes in the convolutional network had the typical size of objects in Atari games built into the system, so we made a simple change to our algorithm: instead of dividing the screen into tiles, we divided it into objects to examine the offsets between objects. But how to find the objects?

We did the simplest thing possible: call all segments with the same coloured pixels an object. If one colour was surrounding another, up to a certain threshold, we assumed the whole object had the surrounding colour and ignored the colour inside. By taking the offsets in space and time of these objects, we obtained a new feature set called Blob-PROST. Figure 5 is a simplification of what we ended up with.

Representation of objects identified for the Blob-PROST feature set
5. Representation of objects identified for the Blob-PROST feature set

So how good are Blob-PROST features? Well, they score better than DQN in 21 out of 49 games (43 per cent of the games) with the score of three of the remaining games having no statistically significant difference from that of DQN. Even when an algorithm is compared against itself, we would expect it to win 50 per cent of the time, making our 43 per cent a comparable result.


We started by asking how much of DQN’s original performance resulted from the representations it learns versus the biases already encoded in the neural network: position/translation invariance, movement information and object detection. To our surprise, the biases explain a big part of DQN’s performance. By simply encoding the biases without learning any representation, we were able to achieve similar performance to DQN.

The ability to learn representations is essential for intelligent agents: fixed representations, while useful, are an intermediate step on the path to artificial general intelligence. Although DQN’s performance may be explained by the convolutional network’s biases, the algorithm is a major milestone, and subsequent work has shown the importance of the principles introduced by the research team. The state-of-the-art is now derived from DQN, achieving even higher scores in Atari games and suggesting that better representations are now being learned.

For a more detailed discussion of each of the evaluated biases, as well as of DQN’s performance as compared to Blob-PROST, read our paper: “State of the Art Control of Atari Games Using Shallow Reinforcement Learning.”

What is Machine Intelligence?

Existing at the intersection of machine learning and artificial intelligence, machine intelligence is advanced computing that enables a machine to interact with its environment in an intelligent way.

Amii specializes in the research and development of machine learning technologies, including their application in artificial intelligence.

What is Artificial Intelligence?

Artificial intelligence (AI) is a set of algorithms, processes and methodologies that allow a computer system to perform tasks that would normally require human-level intelligence. AI can appear as a component in a larger system or in the form of a computer application, digital agent or autonomous machine.

What is Machine Learning?

Machine learning is a field of computing science focused around developing algorithms that enable a computer system to independently learn from, and continuously adapt to, data without being explicitly programmed for that data. Machine learning is a crucial component in many artificial intelligence systems.

Where is Machine Learning Used?
  • Recommender systems (e.g. Netflix or Amazon)
  • Contextual web searches (e.g. Google)
  • Intelligent digital assistants (e.g. Cortana or Siri)
  • Game-playing AI (e.g. AlphaGo or Cepheus)
  • Autonomous vehicles
  • Email spam filters
Why is Machine Intelligence Important?

Recently, machine intelligence technologies have experienced a global resurgence due to growing volumes and varieties of data, the utility of this data in training smart systems and an increased awareness of the value of data in providing a competitive edge in business.

Machine intelligence is expected to form the basis for most technological and business advancements for years to come. According to a report issued by McKinsey & Company, technologies that employ machine intelligence will have created over $50 trillion in economic impact by the year 2025.

How Can Machine Intelligence Enhance My Business?

Machine intelligence allows organizations to operate more efficiently and effectively, using data to predict the future and manage the present.

Computer systems with machine intelligence can perform a variety of tasks:

  • Optimize and automate processes
  • Extract and classify data
  • Detect, analyze and predict trends/patterns
  • Enhance interaction with humans/the environment

Want to bring machine intelligence to your business? Visit our Innovation Affiliates program.

Amii News

From AICML to Amii

Introducing Amii

We’re thrilled to introduce you to Amii, the Alberta Machine Intelligence Institute, a world-leading team of machine intelligence researchers housed at the University of Alberta.

We originally started in 2002 as AICML (the Alberta Innovates Centre for Machine Learning), specializing in advanced research and development in the fields of artificial intelligence (AI) and machine learning, together called machine intelligence.

As a research institute based out of the Department of Computing Science, we push the bounds of academic knowledge and develop innovative solutions for some of the toughest challenges facing Alberta and beyond.

With the launch of our new brand, we are reaffirming our commitment to creating and discovering the future of machine intelligence.

Driving Innovations in Research

Our team of 11 researchers conduct advanced research in areas such as reinforcement learning, algorithmic game theory, data science and health informatics, among others.

Many will recognize our team’s contributions to the varied field of machine intelligence.

In 2007, we solved checkers, a long-standing challenge problem for AI researchers, and in 2015, we produced the first AI agent capable of playing an essentially-perfect game of heads-up limit hold’em poker. Through the Arcade Learning Environment, we’ve encouraged researchers around the world to adopt a new challenge problem focused on Atari 2600 games.

Outside of AI challenge problems, we recently launched PFM Scheduling Services, which revolutionizes the way scheduling is done in union environments. We’ve also produced leading-edge innovations toward the diagnosis of tuberculosis and ADHD, and we work to enhance the lives of people with upper-body amputations through intelligent artificial limbs.

Through these and a number of other projects, we continue to push forward fundamental understanding of machine intelligence algorithms, architectures and applications.

Delivering Intelligent Business Solutions

In launching our new website,, we’re also stepping up our efforts to deliver innovative applications of machine intelligence to businesses in Alberta and beyond.

We collaborate with organizations of all types and sizes to develop machine intelligence solutions that meet specific business challenges. And we provide intelligent tools that can enable an organization to predict trends and patterns, analyze and classify data or optimize and automate processes.

We also recently launched our Industrial Affiliate Program, which provides an opportunity for a deeper level of engagement with Amii’s experts. Through facilitated interactions, Affiliates can gain enhanced access to our world-leading researchers and students and also discover the latest advancements in machine intelligence from the international research community.

Whether you’re looking to enhance your business, enable scientific discovery or understand the next wave of advanced computing, Amii can help.

Contact us and discover the future of machine intelligence.