Ferdinand Hingerl has no hesitation about artificial intelligence.
Hingerl is the Chief Technical Officer of Ambyint, a Calgary-based company whose AI-powered platform optimizes production for upstream oil and gas companies. Ambyint’s platform uses autonomous algorithms to manage roughly $12 billion in production in North America, reducing the need for manual intervention.
There’s a lot of uncertainty in oil-and-gas production. Weather conditions change and wells become less stable as equipment fails. And that uncertainty comes at a great expense —it can affect not only the bottom line, but also worker safety and environmental impacts. Ambyint’s goal is to reduce that uncertainty.
“What we're doing is really decision making under uncertainty for control in industrial control systems,” he says.
Hingerl says his team wanted to investigate how reinforcement learning (RL) could enhance Ambyint’s platform. They were looking for a partner with RL experience that could support them through that research process, reducing the risk and cost that comes with any new, innovative approach.
Hingerl says he knew exactly who he wanted that partner to be: the Alberta Machine Intelligence Institute (Amii), a world leader in reinforcement learning.
“I read a lot of research papers out of Amii, “ he says.
“We're getting close to something that can actually be deployed. So I said, let's engage with Amii and see if we can actually develop something for the actual use case that we have at hand.
Under pressure
The challenge that Ambyint and Amii set to tackle was the automation of plunger lift wells.
When a new well is drilled into a reservoir, it relies on the natural pressure underground to force the oil or gas up to the surface. But, as the wells age, water and liquid hydrocarbons will accumulate at the bottom of that well, lowering that natural pressure. When it lowers enough, the well is no longer able to function without some sort of intervention.
A plunger lift is one of the most common ways of keeping a mature well working. A plunger is lowered into the well, where water and other liquids collect over the top of it while pressure builds underneath.
Eventually, when the pressure builds high enough, a surface valve is opened. This creates an extreme difference in pressure, which fires the plunger back up the well, sort of like an industrial-sized bottle rocket. The water and other liquids on top of the plunger are pushed out with it, allowing the reservoir to force the gas to the surface once again for a period of time. When the pressure drops past a certain point, the valve closes, and the plunger drops back down so the process can be repeated.
Plunger lifts are very commonly used in the oil and gas industry: they are cost-effective and don’t need external power to use, as they rely on the natural pressure found underground. But while the idea of a plunger lift is simple, actually operating one is not. The setpoints — the pressure thresholds that determine when the plunger should go up and down — are tricky to determine. The conditions underground can change rapidly, and the proper setpoints can easily change in a matter of days.
And getting the setpoints wrong can cause significant problems.
“If you don't react fast enough to changes that make your current choice of set points invalid, then your plunger will get stuck at the bottom,” he said.
A stuck plunger means a dead well, at least until the situation can be fixed. This is usually done by a venting event, where the well is closed, and the pressure is increased until the plunger can be pushed back up again. Hingerl says the goal is to avoid these stuck wells as much as possible: fixing them can be dangerous to the workers and is costly. As well, venting events can release a lot of methane, a powerful greenhouse gas. Methane in the atmosphere traps heat very well, meaning it has 28 times the impact on climate change compared to an equivalent amount of carbon dioxide.
Ambyint’s current automation platform can already reduce these venting events by 60-70%, Hingerl says. But they wanted to explore how reinforcement learning could push that efficiency even higher, leading to lower costs, greater worker safety and less environmental impact.
The problem proved to be a challenging one. Reinforcement learning projects often involve some form of simulation - creating a digital version of the problem so that RL models can be tested much more efficiently than in the real world. However, a good simulation requires a lot of data, which is hard to come by in oil wells. Pretty much all of that valuable data is kilometres underground where the reservoir sits.
“Everything is literally like five or six kilometres down. At the surface, you have absolutely no measurements of anything that happens in the subsurface,” he says.
What's going on underground?
Dealing with that uncertainty required a very specific, collaborative approach. One that combined Ambyint’s deep knowledge of oil well dynamics with Amii’s world-leading expertise in reinforcement learning. Rather than just building the model isolation, Amii’s machine-learning residents worked hand in hand with members of Ambyint’s technical team. The result was a collaborative, iterative process that not only worked on the problem but also raised the Ambyint team’s understanding and familiarity with reinforcement learning and what it can do.
“We met with their technical team every week and talked through the different approaches. It is very collaborative in that sense,” said Dave Staszak, Lead Machine Learning Scientist with Amii’s Advanced Technology team.
That’s part of the goal: helping them think through the problem and learn about reinforcement learning.
Ambyint and the Amii Advanced Tech team ended up trying a few different RL approaches to improving the setpoints that determine when a plunger is dropped or released.
The first approach was a direct offline RL method, which used historical data on past setpoints to make predictions for the future. While it showed some success, being used on a live well meant it was limited in the actions it could take to explore methods for greater efficiency.
Instead, Amii’s ML scientists tried to simplify the goal. Instead of using RL to try to build a long-term plan for what the setpoints should be, the model would monitor the current conditions in the well and react only to what was going on at that moment.
Finally, they tried another clever solution, based on physics. This model combined historical data of previous setpoints with a set of rules based on the known physics of liquids, gases, and pressures. It isn’t quite the same as a simulator, which would be based on the actual recorded conditions of a reservoir. But it is more of a best guess.
This model provided the most accurate predictions. Although it was still limited by not being able to check the actual conditions going on underground.
“You’re talking about what’s going on in a hole kilometers into the ground. So it’s very crazy and chaotic. People spend all their careers trying to figure it out,” Staszak says.
RL and the future of industrial control
In the end, the RL models offered some accurate results, but they were not as reliable or reactive to changes as Ambyint required, and they didn’t provide more efficiency than Ambyint’s current automation platform.
Hingerl says that the RL solutions they developed with the Amii team might not be ready for production right away, but the project did yield something invaluable: knowledge. Having gone through the process with the Advanced Tech team, he says Ambyint gained invaluable experience with reinforcement learning.
“Amii provides a complete fresh set of eyes, experts. And then ask a lot of questions about why you actually do things. So, we look at it our way. Amii looked at it in a very different way, and together we complemented the picture very nicely."
While the specific project might not have yielded immediate results, Hingerl says that collaborating with Amii did the important work of training his team on the particulars of reinforcement learning and how it can be used in future projects. Just as the project was focused on reducing uncertainty in oil wells, Hingerl says working with the Amii team reduced a lot of the uncertainty when it came to working with reinforcement learning, reducing the time and resources needed to explore how RL could be used in Ambyint’s work. And now, his team is better equipped to use it to their advantage in the future.
The company will continue to do research into how reinforcement learning may be integrated into its automation platform. And he is convinced that will lead to large benefits as the technology matures and some of the challenges are worked out.
Hingler says “I believe in this. This is the future. It just needs to be refined more on the research side.”
Delivered by Amii and supported by Prairies Economic Development Canada, this project catalyzes AI adoption across the prairies. Through this initiative, Amii equips small-and medium-sized businesses across Alberta, Saskatchewan, and Manitoba with the critical technical expertise to thrive in an evolving market. By driving regional growth, this partnership establishes the prairies as a leader in AI-driven innovation—accelerating local economic development, creating opportunities, and strengthening Canada’s overall AI ecosystem.
)