ML Process Lifecycle – Part 3: Project Expectations vs Reality

This is the first part of a three-part series on the ML Process Lifecycle. Read Part 1 and Part 2.

We’ve covered what the ML Process Lifecycle (MLPL) is and why it’s important, and looked at the framework and its key aspects. In this third and final post of the MLPL series, we’ll take a look at what the framework looks like in action!

It’s a given that whenever we take on a project, we want it to go smoothly. Unfortunately, the reality is that most (if not all) projects will run into obstacles and challenges. This is especially true for exploration tasks.

Expectations vs. reality

This is what we expect:

A graphic demonstrating what most people expect ML projects to look like - a simple progression from "Objectives" to "ML Solution"

But this is usually what we get:

A graphic depicting a real Amii ML Project, moving from "Objectives" through several lifestyle switches, hitting the "ML Solution" after 14 iterations

As illustrated above, we frequently have to pump the brakes and go back, either to one of the previous phases or to one of the modules in the same phase. We refer to this as a lifecycle switch. A lifecycle switch forces you to revisit past stages in order to address new information and challenges as they come up.

That’s also the reason why we call this a “cycle”; one step follows the next, and each step affects all the others, especially the ones downstream. This means that if you go back and change or re-do a step, you’re going to have to revisit the subsequent steps, because they’re probably going to change, too. 

Framework in action

The below example shows the lifecycle switches of a past Amii ML project:

Real Amii ML Project with Lifestyle Switches

While phase one went smoothly (the first time around), in phase two we identified that we had data for only one season of the year. Temperature affected the business question we wanted to answer, so in order to have an accurate year-round model, we needed data from all four seasons. Once the new data was collected, we had to do most of the data analysis over again, such as checking assumptions, aligning different files, and cleaning the data. 

You can see in the fifth line that after we had that dataset in a good place for building our machine learning model, we ran into another lifecycle switch. The organization decided that the original problem they had defined was no longer a business priority; while this may seem odd, it’s actually quite common for business objectives to change in exploratory projects, and in this case it made sense to start from the beginning to pursue a business objective that provided greater business value.

As a result, the whole project was switched back to phase one to iterate again, and went through several more lifecycle switches before eventually concluding with a desirable ML Solution.

Reasons for a lifecycle switch

As you can see in the example above, there are a number of reasons why a lifecycle switch might be necessary. 

For example, there may be business reasons, such as the defined business problem not aligning properly with business goals, a change in the business objectives of the organization, or insufficient business value to justify the expense of an ML project. 

Data issues often result in lifecycle switches as well, such as insufficient data quantity or quality, the dataset to hand not addressing the defined business problem, or the historic data not being an accurate model of the current situation. And sometimes the ML development itself goes awry, or doesn’t give accurate enough answers. 

Lifecycle switches are part of the process of an ML project, inherent in its iterative nature. It is essential that organizations understand going into a project that these switches are going to happen.

Final words

We hope you have enjoyed our series on the MLPL! We know that ML projects are challenging; they’re iterative, exploratory, and often have unexpected obstacles. That is why having a framework, such as our MLPL, can be very helpful. It gives technical experts, non-technical managers, and other stakeholders such as clients and boards a clearer idea of what is involved, and it gives a framework and common vocabulary to communicate about value, objectives, expectations and obstacles.

Amii’s MLPL Framework leverages already-existing knowledge from the teams at organizations like Microsoft, Uber, Google, Databricks and Facebook. The MLPL has been adapted by Amii teams to be a technology-independent framework that is abstract enough to be flexible across problem types and concrete enough for implementation. To fit our clients’ needs, we’ve also decoupled the deployment and exploration phases, provided process modules within each stage and defined key artifacts that result from each stage. The MLPL also ensures we’re able to capture any learnings that come about throughout the overall process but that aren’t used in the final model.

If you want to learn more about this and other interesting ML topics, we highly recommend Amii’s recently launched ML Technician Certificate Course. Visit the Training page to learn about all of our educational offerings.

Latest News Articles

Connect with the community

Get involved in Alberta's growing AI ecosystem! Speaker, sponsorship, and letter of support requests welcome.

Explore training and advanced education

Curious about study options under one of our researchers? Want more information on training opportunities?

Harness the potential of artificial intelligence

Let us know about your goals and challenges for AI adoption in your business. Our Investments & Partnerships team will be in touch shortly!